feat: config files for backend options

[psschwei]

Misc PR

Type of PR

• Bug Fix
• New Feature
• Documentation
• Other

Description

• Link to Issue: Feature Proposal: Configuration File Support #389

(still working on this, but figured I'd post what I've got so far)

This PR introduces a user configuration system for backend options. It adds a new mellea/config.py module and CLI commands (cli/config/) that allow users to configure AI backend settings (like API keys, model selections, and provider preferences) via config files rather than hardcoding them. The feature includes comprehensive documentation, integration with the existing session management, and extensive test coverage.

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code as added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

[github-actions]

The PR description has been updated. Please fill out the template for your PR to be reviewed.

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

[jakelorocco]

I think this is really interesting; will have to think a bit more about how this works. A few questions that I wasn't able to easily determine:

1. How do I create a new session with no model options if I have model options configured in my config? It looks like it always uses the config.
   a. Same for backend kwargs.
2. For the model opts, we should probably also utilize a hierarchy if we go this route: a. Allow model opts to specified for any backend, force these to use the special mellea syntax for generic model options (ie @@@temperature@@@).
   b. Allow model opts to be specified per backend; that way if you change the backend, you can still get defaults.
3. This limits the contexts that you can set in your defaults. How would you set a custom context type?
4. Does it correctly pickup that it's not the default if you explicitly pass in "ollama"?

I like that this allows sharing configurations a little bit more easily; but I'm also uncertain since it likely hides some behavior / might make it harder to debug. At the very least, I think we should we add some logging to start_session() that logs that it's using a config.

I'm also thinking that maybe this should be added as a separate function (something like session_from_config that defaults to the start_session behavior if no configs are specified? But I think that would depend on the answers to some of the questions above as well.

(I might also just be too stuck on our current defaults / how it works 😅 )

[psschwei]

I think this is really interesting; will have to think a bit more about how this works.

I'm also happy to make changes as we go, especially as we work through the details.

1. How do I create a new session with no model options if I have model options configured in my config? It looks like it always uses the config.
   a. Same for backend kwargs.

Speaking of which 😄

That's a good question. Are you thinking about having a way to override the global config? Assuming no global config, the intention here was to look for a local config and if it doesn't exist then just use the defaults.

2. For the model opts, we should probably also utilize a hierarchy if we go this route: a. Allow model opts to specified for any backend, force these to use the special mellea syntax for generic model options (ie @@@temperature@@@).
   b. Allow model opts to be specified per backend; that way if you change the backend, you can still get defaults.

Interesting... I was only thinking about this at a backend level, but being able to specify options across backends (like temperature) is really intriguing.

3. This limits the contexts that you can set in your defaults. How would you set a custom context type?

I believe you could still pass the custom context through the ctx variable, e.g. with start_session(ctx=MyCustomContext) as session: .... Which I think is ok (?)

My initial thought was that the only things that would be configurable would be named parameters like backend and model. The use case I had in mind was something like this: I'm building an application my laptop, where I use ollama. But then I want to run it somewhere else, where I'd need to use (say) watsonx. I don't want to have to rewrite the code (or hack in sed commands to rewrite things), so being able to set as configuration would be ideal.

4. Does it correctly pickup that it's not the default if you explicitly pass in "ollama"?

If it doesn't, we can make it 😄

I like that this allows sharing configurations a little bit more easily; but I'm also uncertain since it likely hides some behavior / might make it harder to debug. At the very least, I think we should we add some logging to start_session() that logs that it's using a config.

Adding logging makes sense. If we logged out the full list of options being set, would that provide full transparency, or is there something else you think would be obscured? (you know this codebase much better than me)

I'm also thinking that maybe this should be added as a separate function (something like session_from_config that defaults to the start_session behavior if no configs are specified? But I think that would depend on the answers to some of the questions above as well.

Using a separate function would sort of short-circuits having a model config though, because you'd still need to make a code change in order to use the config.

(I might also just be too stuck on our current defaults / how it works 😅 )

No worries! I kind of went at this a bit backwards (wrote the PR before the issue), so good to work through all the details here.

[psschwei]

I made a few updates to address some of the concerns:

• Dropped the global config and went local config only. This should simplify the question around defaults, essentially if mellea.toml exists, it's used; if not, then use the defaults
• Fixed explicit backend detection via _UNSET sentinel pattern
• Config now supports generic + per-backend model options
• Logs when using a config file