/ /
🐍
sudopython
Search
Notion
Drag image to reposition
🐍🐍
sudopython
Last update
Feb 18, 2021
Tags
python
talk
Time spent (hours)
40
On the road to babelia, I discovered several things both with Scheme and Python. This log / talk is quick feedback on what I discovered.
Proper-noun spell-checking benchmark
levenshtein C
levenshtein distance with cutoff
fuzzywuzzy
hamming distance
okvs + bbkh + fuzzywuzzy
Création de l'index
1.
key = bbkh_pack(proper_noun)
2.
okvs.set(key, b'')
Requête sur l'index
key = bbkh(query) counter = Counter() for key, _ in okvs[key - limit * alpha:key + limit * alpha]: bbkh, proper_noun = bbkh_unpack(key) counter[proper_noun] = fuzzywuzzy.ratio(query, proper_noun) print(counter.most_common())
Python
Recherche textuelle
Création de l'index
words = document.translate(string.punctuation, ' ').split()) stems = set(stem(word for word words) for stem in stems: db[stem].add(document)
Python
Requête de l'index
query = valid(query) seed_stem = select_seed(query) # consider only positive terms candidates = db[stem] counter = Counter() for candidate in candidates: score = compute_score(query, candidate) counter[candidate] = score
Python
Optimisations
Pool.map
Pool.map_unordered
Amazon Web Service
🗣
Pop-culture
🤑
Pricey
🙈
Hidden costs
💰 AWS vs. System76
For three years, rent vs. part warranty.
AWS: $67,978.79 ($23,941)
48 vCPU AMD
192 G RAM
1 TB SSD
System76 : $7,003
2 * 12 cores Intel (2 * 24 threads)
2 * 96 G RAM
2 * 1 TB SSD
⛈️ AWS vs. Hetzner
For three years, VM vs. dedicated.
AWS: $67,978.79 ($23,941)
48 vCPU AMD
192 G RAM
1 TB SSD
Heztner: €10,188
32 cores AMD (64 threads)
192 G RAM
2 * 1 TB SSD