Skip to main content

Jan Halfar

While building this website and integrating https://docsearch.algolia.com and evaluating another solution by a large company in parallel I could not help to search github and the web for the current state of search engines and search related services.

Since I had done the same thing about a year ago, I was surprised to see how quickly things are moving atm.

Algolia

I was blown away by the quality of https://www.algolia.com and I wish it was open source, but I guess, we all have to make a living ;)

To see how awesome a web (search) interface can be check out https://www.lacoste.com/us/#query=red%20jackets%20for%20men

Apart from that the UI/UX of their backend tools is fantastic.

Elastic

When it comes to https://www.elastic.com I am a bit nervous about the future of the licensing, despite the fact, that I understand their motivation. At the same time the https://opensearch.org does not seem to be an ampty threat.

typesense.org

I do not know, who was hiding under a rock, but I had not seen https://typesense.org before and they certainly have a bold claim: "The Open Source Algolia Alternative" / "The Easier To Use ElasticSearch Alternative"

When looking at https://github.com/typesense/typesense/graphs/contributors it seems, that Kishore Nallan has been working on this for a while. Unfourtunately I do not really see a lot of external contributions, C++ does not seem to attract a lot of contribution.

MeiliSearch

This Rust project https://www.meilisearch.com/ seems to be picking up speed and is definetly on the test short list. It is a fresh codebase with siginficant open source contributions and certainly will attract new developers with Rust and a modern architecture.

Go eco system

Obviously we are very interested in Go powered software and there are a few notable projects. ATM I do not see anything elastic or algolia like, that would be really mature.

bleve / bluge

Marty Schoch seems to be the man when it comes down to text indexing libraries in written in Go and bluge seems to be THE library, that is solid and modern, when implementing text search in your Go application.

https://github.com/blevesearch/bleve https://github.com/blugelabs/bluge // next iteration of bleve

projects using bluge

All bleeding edge afaik atm - but definitely good places to look at bluge usage

https://github.com/prabhatsharma/zinc https://github.com/mosuka/phalanx

Look ma I made a vector database

Gotta take a look at this one - will report later

https://github.com/semi-technologies/weaviate

Marko Trebižan

Issue with performance

When building an ecommerce site or an application where performance is a great deal for the users, you need to keep your application fast and responsive. Frontend developers have already many use-cases when the UI becomes laggy and this increases when 3rd party scripts are being included, such as Google Tag Manager or various Live chats (e.g. Intercom).

This does not only influences the users when using the site but also Lighthouse score gets lower which also influences page rankings. So the most naive and easy way for this is to defer loading of such scripts but when you need to get all the data from the start of the application, such tactic is not an option. So what else can we do?

Partytown to the rescue

Developers at BuilderIO created an library Partytown that would allow relocating resources from 3rd party scripts off the main thread. We won't dive into specifics how it works, because they explain it nicely on their GitHub page.

In our stack we use Next.js React framework and we will go through the basic steps that will allow us to include Partytown for Google Tag Manager.

Setup

Partytown script needs to be located inside our application and live on the same domain. Since we're using monorepo structure, we need to copy this script across all our frontend application. For that we used CopyPlugin webpack plugin in our Next.js config file:

config.plugins.push(
...
new CopyPlugin({
patterns: [
{
// we copy script from node_modules partytown package to `~partytown` folder in our package that serves static files
from: path.join(path.dirname(require.resolve('@builder.io/partytown')), 'lib'),
// paths for SSR and client side rendering differ
to: path.join(`${isServer ? '..' : '.'}/static/assets/`, '~partytown'),
},
],
})
);

Partytown's requirement is that it needs to know what script should it load into own web worker. For that we set script type to text/partytown. This will prevent script to load on initial load.

Inside _document.tsx we add this:

<Head>
...
// include Partytown and set custom path due to multiple frontends
<Partytown lib={`${addTrailingSlash(this.props.basePath)}_next/static/assets/~partytown/`} debug={false} />
// tag 3rd party script with partytown type
<script type="text/partytown" src={`https://www.googletagmanager.com/gtm.js?id=${id}`} />
...
</Head>

Results

So now, does it work? We used one of our large Ecommerce sites to test the landing Lighthouse score.

This was before adding Partytown:

Lighthouse before Partytown

Here you can see 2 critical things: almost 1s of total blocking time (TBT) and 9s of time to interactive (TTI).

After we added Partytown, we got this:

Lighthouse after Partytown

Time to interactive went from 9s to 6.1s which is almost 33% improvement and total blocking time was reduce by more than 50%! We were more than impressed how easy it was to improve our performances.

Side note: Both screenshots were compressed using Squoosh App.

Next steps

After successful testing of Partytown for Google Tag Manager script, we are more than interested in trying it out on our other scripts. One important topic will be to test Partytown with other service-worker related libraries and how to use them together.

Jan Halfar

A few years ago we abandoned the previous version of https://www.foomo.org as we did not want to maintain the old wordpress installation and the project was only living in README.md in the repos living under https://www.github.com/foomo .

As things have grown over time we have decided to re-launch a website / cross project documentation.

So welcome back and enjoy the view to the past:

blast from the past