2008-03-20 15:20:22
キーワードの前後の単語の出現頻度の順で並べ替えてみる。
#!/usr/bin/ruby
# vim: set fileencoding=utf-8 :
import sys
import re
word = sys.argv[1]
file = sys.argv[2]
bf = sys.argv[3]
length = len(word)
txt = open(file,'r').read()
txt = re.sub('\x0D\x0A|\x0A|\x0D', '\n', txt)
txt = re.sub('(\n)+', ' ', txt)
txt = re.sub('\t', ' ', txt)
txt = re.sub('<.+?>', ' ', txt)
lines = []
bfwds = []
strt = 1
while strt > 0:
pos = txt.find(word,strt)
if pos == -1: break
left = txt[pos-30:pos]
right = txt[pos:(pos + len(word) + 30)]
bwords = left.split(' ')
bwords.reverse()
if bf == 'f':
rstring = right[length:]
fwords = rstring.split(' ')
if len(fwords[0]) == 0:
fword = fwords[1]
else:
fword = fwords[0]
lines.append([str(pos),right,left,fword])
bfwds.append(fword)
else:
lines.append([str(pos),right,left,bwords[1]])
bfwds.append(bwords[1])
strt = pos + 1
wdict = {}
get = wdict.get
for bf in bfwds:
wdict[bf] = get(bf, 0) + 1
for line in lines:
line.append(wdict[line[3]])
line.reverse()
lines.sort()
lines.reverse()
for line in lines:
print line[2].rjust(25) + line[3] + ":" + line[4]
本当はスクリプトの中に説明をきちんと書くべきなんだろう。wdict={}のところで、キーワードの前あるいは次の単語リストを集計している。その集計値を二次元リストに追加してソートして表示する。やってみると呆気ない。結果は次のとおり。
$ python kwicfq.py moonlight drodr10.txt b w, or whether it was only the moonlight, he never knew. Their spirits:356000 traits that were clear in the moonlight eyed him with absolute apathy:217201 re was glowing redly into the moonlight through the wide door made fo:199239 rafina was moving through the moonlight as though its rays were her s:218946 oks they had given him in the moonlight, and all looked back at him w:355650 o danced, so sparkled. In the moonlight also one makes no unworthy st:19383 merry play of my sword in the moonlight was often a joy to see, it so:19303 longer than man's. And if the moonlight streamed on untroubled, and t:204443 layed languorous music in the moonlight and sang soft by her low balc:44267 ho might be familiar with the moonlight in that shadowy chamber shoul:42272 he sunset grew dimmer and the moonlight stole in softly, as a cat mig:193054 hat now approached him in the moonlight round a corner of the house w:205397 hat had frowned at him in the moonlight when he came here before, fro:355299 grow dimmer, and glide to the moonlight again that streamed through a:218073 elf in a long hall lit by the moonlight only, which was looking in th:216337 ch lies always so near to the moonlight, and was not in front of him :215294 at, if interrupters come, the moonlight is better suited to the play :19175 a sheer wall that even in the moonlight fell into blackness. Rodrigue:38713 . He saw her pass through the moonlight and grow dimmer, and glide to:218029 panish gardens, remembered by moonlight in Spring, for the other end :20838 apiers crossing each other by moonlight begun to gleam in the street :206461 when her lover came again by moonlight had cast them all from her fr:44481 n white, and all shining with moonlight, came Serafina. Rodriguez in :217931 a chamber partly shining with moonlight. "In there," said the man tha:216854 er, lurking along the edge of moonlight and darkness, disappearing an:217555 For some moments the spell of moonlight on sunlight hovered: the air :194333 enough, it was not yet wholly moonlight when cantering hooves came do:350318 night, all drenched in white moonlight, sheltering huge darkness in :121534 rily towards him in the clear moonlight with a sword. Morano was frig:210798 all alone with disaster, and moonlight pouring down, and the black g:122007 all down the long room across moonlight and blackness, came the lady :221914 And choose, my son, rather a moonlight night when you sing under tho:18413次に、キーワードの次の単語で並べ替えた場合。
$ python kwicfq.py moonlight drodr10.txt f w, or whether it was only the moonlight, he never knew. Their spirits:356000 oks they had given him in the moonlight, and all looked back at him w:355650 n white, and all shining with moonlight, came Serafina. Rodriguez in :217931 ch lies always so near to the moonlight, and was not in front of him :215294 night, all drenched in white moonlight, sheltering huge darkness in :121534 layed languorous music in the moonlight and sang soft by her low balc:44267 er, lurking along the edge of moonlight and darkness, disappearing an:217555 all down the long room across moonlight and blackness, came the lady :221914 . He saw her pass through the moonlight and grow dimmer, and glide to:218029 hat had frowned at him in the moonlight when he came here before, fro:355299 enough, it was not yet wholly moonlight when cantering hooves came do:350318 panish gardens, remembered by moonlight in Spring, for the other end :20838 ho might be familiar with the moonlight in that shadowy chamber shoul:42272 rily towards him in the clear moonlight with a sword. Morano was frig:210798 merry play of my sword in the moonlight was often a joy to see, it so:19303 re was glowing redly into the moonlight through the wide door made fo:199239 longer than man's. And if the moonlight streamed on untroubled, and t:204443 he sunset grew dimmer and the moonlight stole in softly, as a cat mig:193054 hat now approached him in the moonlight round a corner of the house w:205397 all alone with disaster, and moonlight pouring down, and the black g:122007 elf in a long hall lit by the moonlight only, which was looking in th:216337 For some moments the spell of moonlight on sunlight hovered: the air :194333 And choose, my son, rather a moonlight night when you sing under tho:18413 at, if interrupters come, the moonlight is better suited to the play :19175 when her lover came again by moonlight had cast them all from her fr:44481 a sheer wall that even in the moonlight fell into blackness. Rodrigue:38713 traits that were clear in the moonlight eyed him with absolute apathy:217201 apiers crossing each other by moonlight begun to gleam in the street :206461 rafina was moving through the moonlight as though its rays were her s:218946 o danced, so sparkled. In the moonlight also one makes no unworthy st:19383 grow dimmer, and glide to the moonlight again that streamed through a:218073 a chamber partly shining with moonlight. "In there," said the man tha:216854もっと手間をかければ、二つ前、二つ先といった単語でも並べ替えられる。
私はPHPでこういったことができるようになるのに、一年近くかかっている。しかも、MySQLと組み合わせないと頻度順の並べ替えができなかった。