Header logo.
small hallucinations
homeyearstagsaboutrss

Week 46 gave me surprises

A few things:

Handling date and time is more complicated than I expected. python-dateutil is quite helpful when you need to parse a datetime string.

A pitfall

I tried updating a field in a MongoDB document using MongoEngine. The value of that field was an embedded document that contained four sub-fields. I intended to change the value of subkey2. But the entire embedded document was overwritten.

 1{
 2    key1: value1,
 3    key2: value2,
 4    key3: {
 5        subkey1: subvalue1,
 6        subkey2: subvalue2,
 7        subkey3, subvalue3
 8    }
 9
10    # After updating, this field instead became this...
11    # ...and this is not what I wanted
12    key3: {
13        subkey2: newvalue
14    }
15}

I'm quite glad I caught this potential bug early on. And indeed, smart people out there have figured it out.

Baffling decimals

I was trying to check if the submitted value and the value found in MongoDB are the same. This field was defined as a decimal.

In order to try it out, I artificially submitted a JSON object that contained a number with two decimal places (69.30) and the value I saved in MongoDB was 69.3 of decimal type (or I thought so).

But Decimal(submitted_value) == Decimal(mongodb_value) returned False.

Printing out the two values, I got:

1# submitted_value
269.30 
3# mongodb_value
469.299999999999997157829056...

The reason of this turned out to be: Although I defined this field to be a DecimalField with precision=2 in MongoEngine, the value actually stored in MongoDB was still a Double.

And a float number is supposed to do this.

So much fun in week 45

These are the fun I had in week 45, 2021.

Djongo

I ended up in a situation where I decided to use Djongo to bridge MongoDB and Django.

MongoEngine team made a Django connector. But it does not seem to be actively maintained. So they pointed to Djongo and described it as “promising”. (Spoiler: If something is “promising”, the universe hasn't decided whether to resolve it or reject it.)

It seemed as if the only thing you needed to do is go to settings.py and change the database engine to Djongo like so.

 1DATABASES = {
 2    'default': {
 3        'ENGINE': 'djongo',
 4        'NAME': '<db_name>',
 5        'ENFORCE_SCHEMA': True,
 6        'CLIENT': {
 7            'host': '<conenction_url>'
 8            }
 9    }
10}

If you set enforce schema to True, you'd need to run migrations like dealing with a relational DB. Setting this parameter to False would save you some hassle.

Then an error popped up saying a certificate was required. (SSL: CERTIFICATE_VERIFY_FAILED) Djongo's documentation mentioned in passing that you could use arguments allowed in pymongo.MongoClient().

In this case, useful arguments were ssl, ssl_cert_reqs and ssl_ca_certs. But Djongo doc did not tell you how exactly you could do that.

After some trial and error, I did the obvious by directly adding query params to the URL.

  • 'ssl=true&ssl_cert_reqs=CERT_NONE'
  • 'ssl=true&ssl_ca_certs=' + certifi.where()

The first one is less secure. To use an actual certificate, you need to install certifi and go with the second. (There were more trials and errors. And a lot of times, doing the obvious didn't work.)

Maybe I'm overthinking, but...

At some point I wrote this code for an endpoint that expected a JSON object with keys username and password. (Remember destructuring assignment from JavaScript? LOL!)

1def auth_login(request):
2    username, password = json.loads(request.body.decode()).values()
3    # Surely this will work right?

I soon realized nothing prevented the client from uploading something like this:

1{
2    "password": "pa$sw0rd",
3    "username": "lexie"
4}

And since this was not JavaScript, we'd end up with username being pa$sw0rd and password being lexie.

And git ignores case by default

It seems quite a few people don't know git by default ignores case.

I learned this because I once mistakenly named the file for a React component in lowercase then imported it using the capitalized filename.

This could be a problem if you misname a Procfile or a Dockerfile. And you actually don't need to recreate the entire repository like Matt did in this video.

Set ignorecase to false like this and rename the file. A learn this solution (of course) from a note on StackOverflow.

1git config --local core.ignorecase false

Reading Books in Swedish

It took me more than a year to finish reading my first Swedish novel Tio över ett, a teenage romance story set in mining town Kiruna. I got this paperback from a shelf at Uppsala University where people leave their used books. You'd most often see conference proceedings and PhD dissertations – dozens of brand new copies at once – left there.

I also bought the audio version from Bokus.com, because you can download DRM-free MP3 files from there.

Once you have the MP3 files, you can practice dictation with them. It's handy to listen to the recordings using a piece of audio software, such as Audacity or Tenacity.

The idea is: First you read the text and look up new words when you feel like to. Don't worry too much about missing key points in the plot. Think of your brain as a machine learning algorithm. By feeding language data (text and voice) into it, assuming you have an adequate grasp of Swedish grammar, your brain will work out how this language works.

Then you practice listening by writing down every word you hear. This takes enormous amount of time. So you don't need to do the whole book. You probably don't have the time to either. Although I haven't done much dictation on this book, I can feel some slight improvement at listening Swedish.

This week, I started reading Alkemisten by Paulo Coelho. I find more unknown words in this book, as it's set in a different time and geography. But the text flows on beautifully, I just can't put it down.

I use a pencil to underline new words. Look them up when I feel like to. On top of this, when I finish a page, I'd try to summarize the plot in one or two sentences in Swedish. And I'm happily surprised to see I can actually output some coherent Swedish.

Comparing Arrays by Value in JavaScript

I didn't know I didn't know the answer when my friend showed me a simple piece of JavaScript code and asked me what would happen.

1// {1}
2const a = [1, 2]
3const b = [1, 2]
4a == b // what will happen next? 🕶

He said this caused a bug in a project he was working on and it took him hours to debug. Since I knew arrays in JavaScript are objects. And objects are only equal when they refer to the same thing in memory. If you do it as shown in ex. {2}, the result will certainly be true, because x and y indeed refer to the same thing.

1// {2}
2const x = [1, 2]
3const y = x
4x == y // will certainly be true

In ex. {1}, a and b are defined differently. And I felt very clever when I correctly said the code on line 🕶 will produce a false. But —

“If we compare a.valueOf() == b.valueOf(), the result will surely be true!”

— And I was wrong. This would still give you a false!

But my guess was not without a reason.

Take the code below for an example. k1 and k2 are objects. They are not directly comparable. But you can compare their values using valueOf() method. And they are equal.

1const k1 = new Object('wrapping a string literal with an object')
2const k2 = new Object('wrapping a string literal with an object')
3
4k1 == k2 // false; because you can't compare objects to objects
5k1.valueOf() == k2.valueOf() // true; because the values of k1 and k2 are both strings

It came as surprise to me when it turned out arrays, as objects, do not behave like string objects.

The valueOf() method of an array will return an array, which is, again, at the same time an object. Now we are back to this: “you can't compare objects to objects”.

1const m1 = new Object([1, 2, 3])
2
3m1 // [ 1, 2, 3 ]
4m1 instanceof Array //true
5m1 instanceof Object // true
6m1.valueOf() instanceof Array //true
7m1.valueOf() instanceof Object // true

Then I wrote a method that checks if two arrays are equal in value. This method checks recursively the equality in value of nested arrays.

 1// Definition
 2Array.prototype.equalsInValue =  function (that) {
 3  if (! this && that) {
 4    return false
 5  } else if (this.length !== that.length) {
 6    return false
 7  }
 8
 9  for (let i=0;i<this.length;i++) {
10    if (this[i] instanceof Array && that[i] instanceof Array) {
11      if (!this[i].equalsInValue(that[i])) {
12        return false
13      }
14      continue
15    } 
16    if (this[i] !== that[i]) {
17      return false
18    }
19  }
20
21  return true
22}
23
24// Test
25const x = [1, 2, [[3], [4, [5, 6], ['hello', 'world'] ]]]
26const y = [1, 2, [[3], [4, [5, 6], ['hello', 'world'] ]]]
27console.log('comparing x, y: ', x.equalsInValue(y))
28// comparing x, y:  true

I ignored the case where objects are nested inside arrays though. Before I began to take into account objects, I found out someone on StackOverflow had already done that many years ago.


On the other hand, comparing lists in Python is rather uninteresting:

 1x = [1, 2, [3], [4, [5, 6]]]
 2y = [1, 2, [3], [4, [5, 6]]]
 3print(x==y) # True
 4
 5x1 = [1, 2, [3], [4, [5, 6], {'hello': 'world'}]]
 6y1 = [1, 2, [3], [4, [5, 6], {'hello': 'world'}]]
 7print(x1==y1) # True
 8
 9class MyObj:
10    def __init__(self, val):
11        self.val = val
12    def valueOf(self):
13        return self.val
14
15j = MyObj([1, 2, 3])
16k = MyObj([1, 2, 3])
17print(j == k) # False
18print(j.valueOf() == k.valueOf()) # True

Creating My Own Static Site Generator

Since I take a lot of notes, rcently I thought I could edit some of my notes and turn them into a blog. I wasn't satisfied with WordPress, because it's bloated. I didn't feel like to get familiar with the settings and configurations of other static site generators either. So I decided to write my own. I named it “Lysekil”, where I visited in late March and loved it there. I published it on GitHub and provided a brief documentation.

The color scheme used in the template was borrowed from the default “Red Graphite” theme of Bear, my favorite note-taking app. (It also happens to be Andy Matuschak's favorite note-taking app. I used to dither over a few other options, including Obsidian, RemNote and RoamResearch. In the end I would always come back to Bear.)

How I made it

I used third-party Python packages to process markdown, generate the Atom feed and highlight code syntax.

Markdown

Notes in Bear are in a plaintext format very similar to Markdown. Writing blogs in Markdown feels natural.

There are two Python packages available to convert Markdown files into HTML: python-markdown and python-markdown2.

I chose the latter because it allowed for eaiser integration with an extra that enables syntactic highlighting for code. It also seems easier to enable extras (or extension) in general with python-markdown2 than python-markdown.

Syntax highlighting

Both packages allow syntax highlighting using Pygments. It takes a lot more effort to enable code highlighting in python-markdown, according to their documentation.

With markdown2, I could enable “footnotes”1 and “fenced-code-blocks” extras with much ease. To my delight, the “fenched-code-blocks” work exactly the way highlighted code blocks work in Bear.

Feed

There are people on the web who call their personal websites blogs without providing an Atom or RSS feed. (I'm opinionated on this. It's not a blog if there's not a feed)

The package I used to create the Atom feed is python-feedgen. It does support both Atom and RSS. But there are tiny differences between the two. I chose Atom over RSS for no particular reason.

One hitch at this step was that Feedgen required timezone info to indicate the time of creation of entries. To make it work, we needed to pass this information using a timezone object.

This timezone object is defined in datetime package. One way you could specify a timezone prior to Python 3.9 was adding an offset to UTC using a timedelta.

Lucky for me, I've upgraded to Python 3.9.5, which provides a zoneinfo package that allows you to specify timezone info using a string, like this: ZoneInfo('Europe/Stockholm'). (The datetime package in Python 3.9.x also makes it easier to manipulate date and time in strings of ISO format.)

Blogs and pages

Blogs (or “notes”) are presented in chronological order and are organized with tags. They are grouped according to quarters of the year. Pages, on the other hand, are not presented in any timeline.

If you look at the source code, you'd see the implementation of this part is not particularly “DRY”.

Styling

I used SCSS to write the CSS sheetsheet. SCSS allowed nesting CSS selectors and that makes it a lot easier to organize your CSS file.

I tried adding animation effects for the header using JavaScript. I relied on window.onscroll event and it was a bad idea.

It did work on my MacBook Pro, iPhone and iPad. But because of the offset of scrolling on the y-axis and the difference between widths of shown elements, this doesn't work when the page is not long enough for certain screens.

When the page is not long enough, as the user scrolls up, there isn't an unambiguously large offset that triggers the change of width that can persist. Not to mention the (at least theoretical) performance issue caused by too many function calls as the page goes up. (This in itself can be talked about in more detail.)

I then tried using { position: sticky } to place the navigation bar on top of the article page. It did look nice. But since it's rather easy to navigate around here, letting the navigation bar take up too much space at the expense of reading experience didn't feel right. This was especially true on a smartphone.

What I did wrong

I began working on this SSG on a whim, without clearly planning out what functions I wanted and how I'd like to achieve them.

At the beginning I thought a single script that converted a bunch of .md files to HTML would do. I ended up with a template that consists of HTML code snippets sprinkled across six helper functions. (If anyone wants to create a new template for this SSG, I would recommend not to. LOL.)

To organize timelines, tags and archive groupings, I ended up creating two classes for content (notes and pages) and three classes for listings (tags, archive groupings and the home pagination). Had I planned it beforehand, with more proper inheritance, the classes would have looked DRY-er.


  1. Titta! Detta är en fotnot. (Look! This is a footnote.↩︎

Emulating Python's zip() and zip_longest() in JavaScript

This is essentially how you emulate the way Python's zip() function works using JavaScript.

1let a = [1, 3, 5];
2let b = [2, 4, 6];
3const mapping = a.map((item, idx) => [item, b[idx]]);
4console.log(mapping);
5// result
6[ [ 1, 2 ], [ 3, 4 ], [ 5, 6 ] ]

What makes this possible is the fact that JavaScript array methods — map(), forEach() and even filter() — take up to three arguments (element, index, array) while they iterate through the array.

In which element is the element that the iterator points to at each step, index is the index of this item in the original array. And it's worth noting the third argument is the original array.

Since map() method of the JavaScript array is more flexible than zip(), we can do more stuff to the iterated item or the array. But if you want to do anything with array, remember the original array changes accordingly. In this case, a:

 1let a = [1, 3, 5];
 2let b = [2, 4, 6];
 3const mapping = a.map((item, idx, array) => { 
 4    array.push(item*2);
 5    return [item, b[idx]]} );
 6console.log(mapping);
 7// Result
 8[ [ 1, 2 ], [ 3, 4 ], [ 5, 6 ] ]
 9console.log(a);
10// Result
11[ 1, 3, 5, 2, 6, 10 ]

As we can see, integers 2, 6 and 10 are pushed to the end of the array.


Python provides a zip_longest() function in itertools.

While zip() cuts off longer lists to fit the shortest, zip_longest() stretches shorter lists to match the longest. Empty slots in shorter lists are filled with a given value. By default this fillvalue is None.

This is how this function works:

1import itertools
2
3a = [1, 3, 5, 7, 9]
4b = [-1, -3, -5]
5c = itertools.zip_longest(a, b)
6
7print(list(c))
8# Result
9[(1, -1), (3, -3), (5, -5), (7, None), (9, None)]

Below I try to emulate this function in JavaScript.

First let's take a look at line 👀. Notably, In order to take indefinite number of arguments, I'm using the ellipsis operator. Unlike unpacking a list using the star operator (*) in Python, using something like ...args, fillvalue=null will raise an error. So I have to use a pair of square brackets to allow for a fillvalue argument. (Let's give it a duck.)

 1let a = [5, 3, 1, -1, -3, -5]
 2let b = [6, 4, 2, 0, -2, -4, -6, -8]
 3let c = ['a', 'b', 'c', 'd']
 4
 5const zipLongest = ([...args], fillvalue=null) => {  👀
 6    const result = []
 7    let i=0
 8    while (args.some( argArray=> argArray[i])) {
 9        const ithColumn = args.map(
10            argArray=> {
11              item = (typeof argArray[i] === 'undefined')  👓
12              ? fillvalue
13              : argArray[i]
14              return item
15            }
16            )
17            result.push(ithColumn)
18            i++
19    }
20    return result
21}
22
23console.log(zipLongest([a, b, c], fillvalue="duck"))
24
25// Result
26[
27  [ 5, 6, 'a' ],
28  [ 3, 4, 'b' ],
29  [ 1, 2, 'c' ],
30  [ -1, 0, 'd' ],  🙄
31  [ -3, -2, 'duck' ],
32  [ -5, -4, 'duck' ],
33  [ 'duck', -6, 'duck' ],
34  [ 'duck', -8, 'duck' ]
35]

Then let's move to line 👓. Empty slots in the JavaScript array are considered undefined.

But undefined and other falsy values, eg empty strings and 0, all evaluates to false. This will confused this ternary conditional. As a result we will get [ -1, undefined, 'd' ] on line 🙄.

A better way is to check the type of item as we iterate. However, a wrong way to do it is using typeof nonExistentValue === undefined.

Although the nonExistentValue is indeed not defined, typeof returns "undefined", which is a string. And this string is a string. It is by no means equal to the reserved word for non-existent things. Thus the condition we used on line 👓.