Multi chat context

[mvcrouse]

Purpose:

• We have an agentic workflow where there an agent has a set of submodules (i.e., nodes) available to complete a task
• We would like to use a single Mellea session to encompass all of the nodes (rather than having completely separate Mellea sessions for each of the nodes)

Problems:

• A single Mellea session has a global context, thus, if each node has separate system prompts, each system prompt would be visible to the other nodes
• Both the input and output to aact are automatically added to the context, hence if the output of a node is to serve as the input to another node, this is challenging as it will be added twice (once as output and once as input)

Solutions:

• Add labels to each context node that allows for control as to how much of the context is visible on a particular action
• Allow for null actions where generation occurs from the existing context and the input (which is None), is not added to the context.

Here's an example of the Context that is used for generation in each node in a multi-node tool calling setup with the proposed solutions

Thought Node:

[
    {
        "role": "developer",
        "content": "You are a thinking agent. You will be provided with a prompt and a list of tools.\nYou must plan how you will solve the tool calling problem with tools.\nMention specific tools and arguments to those tools in your plan\n\nYou have access to the following tools.\n\n{\n    \"name\": \"calculator\",\n    \"description\": \"Perform basic math operations\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"expression\": {\n                \"type\": \"string\"\n            }\n        },\n        \"required\": [\n            \"expression\"\n        ]\n    }\n}\n\n{\n    \"name\": \"get_current_weather\",\n    \"description\": \"Get the current weather\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n            }\n        },\n        \"required\": [\n            \"location\"\n        ]\n    }\n}"
    },
    {
        "role": "user",
        "content": "What is (42 * 19) - 7 and whats the weather in NY?"
    }
]

Tool Calling Node:

[
    {
        "role": "developer",
        "content": "You are a tool calling agent. You will be provided with a prompt and must respond with tool calls.\n\nTo call a function, please respond with JSON for a function call. Respond in a JSON format like {\"name\": function name, \"parameters\": {\"arg1_name\": arg1_val, \"arg2_name\": arg1_va2, .. etc}. Do not use variables. If multiple function calls are needed, separate them with a ';' semicolon\n\nYou have access to the following tools.\n\n{\n    \"name\": \"calculator\",\n    \"description\": \"Perform basic math operations\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"expression\": {\n                \"type\": \"string\"\n            }\n        },\n        \"required\": [\n            \"expression\"\n        ]\n    }\n}\n\n{\n    \"name\": \"get_current_weather\",\n    \"description\": \"Get the current weather\",\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n            }\n        },\n        \"required\": [\n            \"location\"\n        ]\n    }\n}"
    },
    {
        "role": "user",
        "content": "What is (42 * 19) - 7 and whats the weather in NY?"
    },
    {
        "role": "assistant",
        "content": "**Plan**\n\n1. **Calculate the arithmetic expression**  \n   - Use the **`calculator`** tool.  \n   - Pass the expression `\"42 * 19 - 7\"` as the `expression` argument.\n\n2. **Retrieve the current weather for New York**  \n   - Use the **`get_current_weather`** tool.  \n   - Pass the location `\"New York, NY\"` (or `\"NY\"` if the tool expects a shorter form) as the `location` argument.\n\nAfter obtaining both results, I will combine them into a single answer for you."
    }
]

=== 17:44:29-INFO ======
SUCCESS
  0%|                                                                                                                                                    | 0/3 [00:04<?, ?it/s]
mellea.Message(role="assistant", content="{"name": "calculator", "parameters": {"expression": "42 * 19 - 7"}};{"name": "get_current_weather", "parameters": {"location": "New York, NY"}}", images="[]")

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🔴 Enforce conventional commit

This rule is failing.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

[nrfulton]

I think what you are describing is best achieved with a custom context type. We could call it a LabeledContext or HeapContext and give it a principled semantics^[1].

A note on what Sessions are: Contexts are immutable. A MelleaSession is morally just a convienance wrapper around a mutable ref to the latest context. (It also carries the backend so you don't have to thread that through your act calls).

If you want to be able to refer to previous contexts, it might be easier to simply forgo sessions and use the functional style:

ctx_0: Context = ...
mot_1, ctx_1 = b.act(ctx_0, ...)
mot_2, ctx_2 = b.act(ctx_1, ...)
mot_2, ctx_2 = b.act(ctx_2, ...)

representing the trajectory:

ctx_0, mot_1, ctx_1, cot_2, ctx_2, ...

This trajectory need not be a line, of course, it could be a tree or any acyclic DAG.

Additionally, Contexts can be overidden to provide custom "views" on the list of things that have happened.

--

[1]: In some earlier version of the codebase I played around with even using the subset of locals() with type Component | CBlock (at the time Span but more or less equivalent) as this type of labeled ocntext (the variable name being the label).

But I'll take a deeper look tomorrow AM.