How does our brains work?
This is a text I sent someone about this video: https://www.youtube.com/watch?v=OP3dA2GcAh8
Interesting video. It makes an analogy between "brain" being like a RAM/CPU and your "online notes database of your ideas" being a SSD.
Here are some of views on it if you're interested:
I see interesting analogies between this and IR. Say you are thinking of something. You can view that as an IR task from your past experiences. E.g. if you are thinking what car to buy, you are probably semi-consciously thinking of your past experiences with different cars, reviews you have read, recommendations from friends, how much money you have saved up to spend on the car, etc. If you are writing notes and searching through them it is kind of like exact matching (bm25) or even sparse retrieval systems that use trained NNs. In this setting you are basically finding the similarity between a document and a query and concatenating them in a string and passing it to your model. This seems very inefficient, and it doesn't seem to what humans do when they think (e.g. in the car example, your brain certainly does not go through all your experiences and tries to find which ones are relevant).
I think the brain works more like a dense retrieval system (aka a representation based system). In this setting you convert documents to a dense vector as you see them and then later when you get a query you also convert it to a dense vector as well and measure the similarity between the query vector and each of the document vectors. Actually you don't go through all documents, you do an efficient nearest neighbour search to find the relevant documents. I think this is what likely our brain does. Converts thoughts into internal representations (vectors), converts incoming queries into internal representation as well (vectors) and then searches through documents that are most likely to be relevant to that topic (nearest neighbour search) to find the most relevant documents through similarity between the internal representation of query and experiences (inner product between query vector and document vector)