Tuesday, February 1, 2022

Splendid

 Moving on in the last CS50 AI lecture(Language), we then look

at information retrieval but still within the 'bag of words' model. 

thus, we are doing term modelling. A three step dive into the issue

illustrates what can be done.


Step 1. We want to know what our text is about, and use simple term 

frequency. The example looks at the corpus of Sherlock Holmes stories. 


We extract the most frequently used terms, sure enought; but it is not very helpful.

english contaons a lot of helper words like the, and , to. These are common to all texts.

So for the next iteration, we will ignore these.


Step 2. The program is fed a list of function words that will be ignored in the request.



Better, but still redundant. 'Holmes' appears everywhere, Getting clever, one will then

use inverse document frequency ie we want those words that are frequent in one

story, but will penalize those that occur in all. the winning formula will multiply

term frequency by inverse document frequency.




Here we are: colonel, horse, moor...

This same code on the Federalist Papers:


Splendid: starting to see what each section might be about!!

                                                                 *     *     *

Mthematically, the whole success of this approach hinges on the fact that the

log of any number at zero is euqual to 1. If Holmes is part of all the stories, then

inverse document frequency will be equal to 0 and one can safely discard that

term.

Below, how log plays out!

The 10 series:






Various log functions:








Monday, January 31, 2022

Jet Stream

 Hopefully, this is the last of the abnormally cold days we've

been seeing in the last while (-20 to -30 C). You dress like a bear

and a spot of cheek that isn't covered screams this is TOO cold!

I've seen reports this is actually due to climate change with the Jet

Stream slowing down and making a more pronounced sine wave.

But that is not the whole story. Below:



Saturday, January 29, 2022

Determined

 Those very determined truckers in Ottawa are ipso facto lobbying the Biden

Administration; not a promising affair. Is this really necessary? Where is the

bureacratic analysis of the situation for cross-border trade!? Granted

fighting the epidemic is top priority, but deconfinement may need some

thoughtfulness...




                                               *     *     *

There is a tragic element in all this: the one thing that pandemic measures have

been aimed at is avoiding close contacts between the unvaccinated. and this is exactly 

what this demonstration seems to be delivering.  Will it be a cauldron of infection

for Omicron; I hope not.  I have bee looking forward to going for a double double

this week - as Quebec restaurants re-open - albeit at a 50% capacity Tim's... Maybe not!!


Of course, natural immunity is also part of the equation. Hard to know who has natural

immunity from a past infection and whether this is adequate faced with omicron... 👳‍♀️


                                         *     *     *

DAY 3

For any of the truckers who might be dismayed at the difficulties

of finding food and drink in downtown Ottawa; you have my full

sympathy. Ottawa is not a normal city with shopping and eats at its core; it

is a government town with civil servants crowding in during the week

but a largely empty hull week-ends.


Bottom line, there is coffee and muffins everywhere till noon, but then not.

(Booze picks up later in the day). One can buy lunch in a line-up; there are actual

restaurants in government buildings that serve lunch to departmental employees.

To find a family restaurant one needs to go toward the suburbs, down Bank street

going South or out to the West. The train station off the Queensway is large, warm 

and has stuff. There are French speaking folks in the East of the City.


On a normal day, downtown Ottawa is a traffic nightmare early in the morning, and 

then around four in the afternoon. The Prime Minister goes to work in a limo with

police escort at nine in the morning because he wouldn't make it any other way...


                                                   *     *     *


Day 4

Real-time map of Ottawa:


https://traffic.ottawa.ca/map/?lang=fr

Monday, January 24, 2022

ZeroF

 It's cold as anything and I'm getting bored. Might dust off my vaccine 

passport to go to the local foodcourt and get a liter of milk, browse the 

cookies...Who knows!?


This is when I start to muse about the Milankovich cycle!










                                                            Wikipedia