Doug Turnbull's Blog

Reciprocal Rank Fusion, while a useful tool, doesnt magically make hybrid search relevant

A notebook showing the real decisions computing search evaluation stats

The lack of objective definition of good search creates huge hazards when creating search, RAG, AI solutions

Avoiding conflict is the death knell of organizations that leads to a lack of progress and careers that implode.

In reality, staff engineers aren't about 'company wide' impact but a system of patronage where managers reward behaviors they value

My GAR slides from Systematically Improving RAG Applications Sept 2024 course

Junior engineers are foundational to whether a team can collaborate and innovate

Finding search queries to improve is harder than you think. Here's one statistical procedure for deciding whether a query really has a problem -- or if its just noise.

Integrating my BM25 pandas search library, SearchArray, into BEIR, in order to embarass myself in public.

With normal/boring stock returns - have GOOG, MSFT, etc,run out of AI+layoff cards to play?

All that lexical search context you need to build that RAG app

MICES (Mix Camp E-Commerce) talk about planning e-commerce search relevance work with fast prototypes

Berlin Buzzwords + Haystack talks about Reddit's Learning to Rank journey

Every team chooses different types of NDCG, choosing your ideal is perhaps the most consequential decision

Groundbreaking and courageous software ideas start by first impressing 3 good friends

Be of service to others and true to your craft to build a great network

Implementing an exponential search in Cython to speed up position intersections in SearchArray phrase search

The actual bottlenecks are the search results we gather along the way

As Yoda would Say - A joke it is not to add slop to a search system.

We need more than dense embeddings in our 'vector' search

Seriously - why do we need all these vector databases? Do we need dozens of them?

How phrase search works in search array by intersecting roaring-like numpy arrays.

Make traditional text search a core part of the Python data stack

Convert VisualVM's profiler output to a format suitable for a flamegraph

Software documentation that doesn't suck needs to exist with the living

Slides from Berlin Search Tech. Meetup describing an alternative way beyond Judgments and NDCG to think about search offline evaluation.

Helping random people, for free, can be one of the best things you can do for your career.

Why do we rely on such a fractured, vendor-dominated database layer in our supply chain? Why aren't we more worried?

Slides from Chicago Search Meetup, discussing real-world tradeoffs in vector search beyond just the benchmarks

Implementing random projections based LSH as a C Numpy function

Badly implementing locality-sensitive hashing as a vector search solution... for science, edification, 💩, and giggles.

Software engineering is about designing the right feedback loop(s) with limitted resources.

Search orgs fail because teams get stuck in functional silos rather than empowering their peers

Visibly failing, learning, and discovering first principles is how you have real influence on a field

Navigating between hyperfocus/executing vs perspective/information-gathering mentals states is f*cking hard.

Where to get started, next steps to take, how to evolve search from 0 to 1.

Feedback is the lifeblood of getting better, but be careful who you accept feedback from.

Idiot proof "git cd" command to cd to repos in your project dir with fuzzy matching and tab completion.

You can get started improving search relevance without labels and judgments. Which is an imperfect model anyway.

In this post, I have very weak, uncertain labels of relevance / not. However, in aggregate, they may be able to help us make strong determination on the importance of ranking signals.

If you know u and v's dot products to A1...An can you reconstruct u.v?

Taking things up exactly one notch, from one shared reference, to two to estimate a dot product.

Given a reference vector `A`, where we know `u.A` and `v.A` what can we say about `u.v`?

Finding the probability of a dot product between two vectors lets us quantify how much information is in cosine similarity.

What is vector search and why all the sudden are we talking about it?

Ninety degrees isn't particularly special in 2D, but 3D and beyond, it's the expected angle between two unit vectors.

ChatGPT unlocks information from the Web, and away from sites that abuse their users attention with spam and writing that targets Google, not humans.

Index some documents, provide some queries, ChatGPT will tell you the most relevant documents for those queries

Fred works in marketing in New York City and enjoys running. He lost his job at the beginning of the pandemic.

With algorithm development, naive solutions provide a crucial reference implementation for your testing.

Aliases, etc that have made rebase-based workflows in Git much less advanced feeling.

Experiments are to search relevance correctness as unit tests are to code correctness. By definition they're a broken but nescesarry defition of the problem we need to get started.

No need for local setup to play with Elasticsearch from a Jupyter notebook - just use Bonsai + Colab!

Let's explore this key bias in search systems towards the old algorithm and how to overcome it!

Analyzing the plausibility of guessing relevance judgments from runs in the VMWare Zero Shot Kaggle Competition

Can we simulate the likely search relevance labels just from knowing which results shifted and the outcome of an A/B test?

Work with amazing people you love collaborating with, the rest (mission, purpose, etc) falls out from that.

Reimplementing LambdaMART in Python for endless tinkering and learning

LambdaMART directly optimizes whatever search relevance ranking metric matters to your business. This article details how this neat machine learning trick works to target what matters most to your product

Contrasting how each language handles iteration helps understand how to work effectively in either.

In this article: we assume users review every search result. So we need to find that sweet spot when we get to look reaaalllly fricken smart and declare, with confidence, "we have nothing else that matches your query".

Using Pandas to compute Mean Reciprical Rank using the MSMarco Dataset

Judgment lists prevent search whack-a-mole. They provide a safety net for search, allowing you to innovate quickly on relevance with a high degree of confidence.

High end technical consulting is a fantastic thing for you to do mid career. It helps you build a personal brand, deepen soft skills, and focus on challenging technical problems. Why and why not you might want to take this step.

Twitter gives you an illusion of influence over political events. In reality, it meaninglessly fiddles our energy away. Doing our duty requires real work in the real world.

What I know so far on getting cheaper delivery in Charlottesville that avoids Grubhub's shenanigans and supports local restaurants

Add friction to your twitter login to keep yourself sane.

Write to grow closer to the truth. Not because you have all the answers, not to get page views or win internet points. Instead write to broach a point of view and test it against your audience's norms and points of view.