Why Auto Memories Make LLMs Biased

Technology

| Speech to text transcript in Apple Notes.

Ok so today I'm gonna talk about the concept of the context window we may not always know what's inside for example if you ask a large language model a question it will tell you that it saved to memory. What does this mean? It doesn't really have a memory it's just that whenever it thinks it may need it all injected into the conversation.

Okay there is the system prompt, there is your customer instructions, there is the results the search results or tool calling results, there is the thinking results and that's all part of the context window. LOM interface is very a lot if we take a look at ChatGPT's interface we're just presented with an input box but customer instructions are hitting the system prompts hidden temperature is hidden when you ask a question you'll see that it's searching for or thinking and after it's done it's hidden search results are hidden.

This matters because LLMs don't really have an opinion each next token is determined by the previous or by the content of its context window.

Model performance can degrade with context size and there is a limit to the context window. Claude for example has a limit of 1 million tokens well the context depends on the model you're running. There are other cost-saving tracks we use in LLM interfaces like the sliding window if you remember when we used to have ChatGPT 3.5 it used to forget things a lot because as the conversation becomes longer than we no longer we are no longer able to fit in as many things into the window. And the way LLMs currently work is every time you submit our request all the previous content has to be sent again.

The reason why this is important to know is if we have control over the context window I know what's inside then you know what response is to expect. But if you have for example memories, then you don't really know what's inside. Like you could tell something it will save it but then you change your opinion on the thing and then the next time you ask it and it's related to it to your memory it will pull it back into the context window now LLMs tend to agree with you so it will reinforce your existing belief in our way.

Ask an LM why something is good and then you wanna invert the question and ask it why it's bad it will give you a compelling reason for both.

Similarly, if you have a long conversation it will tend to reuse things in the conversation so if you ask a question it will try and include things from the context like you could discuss a random book and then ask for the 10 the best books of all time and they will try and include the book you just discussed. But with memories it will carry this over to a different conversation no matter the context.

This could be useful, but it is not useful all the time because it doesn't know when it's appropriate to include or not.

In conclusion, it's just next word prediction so be careful with it. It has a bias towards agreeing with you and context generally shapes the answer you get in response. It doesn't have a memory.

So, if you talk about cars a lot and then you want to move to a diffefrent topic in a new conversation it's going to keep bringing cars into the conversation. Are you gonna try and involve cars with every conversation? Like if you ask? For example, in a different context it will give you carbon related to the examples if you say top 10 best books is going to include car related books if you ask it what TV shows to watch some of them are gonna car related.

The interface doesn't really make it clear. If you scroll up a conversion in a chat, then you know why but if it's a memory, it's a lot harder to know if it's obvious. This is fine if you know it's being included, but the interface matters a lot because you don't know.

I personally always like to start clean, so no memories and no long conversation because performance degrades with input length.

This is the best way to use it as to edit the frequently added conversations. You can go back and clarify your intent like if you ask it if if it's an answer if it means if if the AI misinterpret what you said what you said edit the message and add your reply. And just keep editing it to add your reply and you end up with a much better answer. If you want the best possible answer then take all of this. Take all of your prompts and put it into one big prompt in the new chat and that's when you get the best response. Because now it's fresh general purpose AI that can handle your request without any external influence.

Email Newsletter
Get updates sent to your inbox.