feat: migrate from Granite 3 to Granite 4 hybrid models by planetf1 · Pull Request #357 · generative-computing/mellea

planetf1 · 2026-01-26T12:05:48Z

Migrate from Granite 3.x to Granite 4.0 Models

Type of PR

Bug Fix
New Feature
Documentation
Other

Description

Link to Issue: Closes feat: Update tests & examples to use granite4 #344

Summary

This PR migrates Mellea from Granite 3.x to Granite 4.0 hybrid models across all backends, tests, and documentation. Note: HuggingFace tests remain on Granite 3.3 due to adapter availability constraints (see below).

Changes

Model Definitions (`mellea/backends/model_ids.py`)

Added Granite 4 hybrid model identifiers:
- IBM_GRANITE_4_HYBRID_MICRO (granite-4.0-h-micro)
- IBM_GRANITE_4_HYBRID_TINY (granite-4.0-h-tiny)
- IBM_GRANITE_4_HYBRID_SMALL (granite-4-h-small)
Restored IBM_GRANITE_4_MICRO_3B with per-backend model selection (Ollama: MICRO, Watsonx: SMALL)
Marked Granite 3 models as deprecated (kept for backward compatibility)
Added vision model: IBM_GRANITE_3_3_VISION_2B

Backend Updates

WatsonxAIBackend: Default model → IBM_GRANITE_4_HYBRID_SMALL
All other backends: Use Granite 4 hybrid models in tests

Test Updates (19 files)

✅ Migrated to Granite 4:

test/backends/test_watsonx.py
test/backends/test_ollama.py
test/backends/test_litellm_*.py (3 files)
test/backends/test_vllm*.py (2 files)
test/stdlib/components/*.py (8 files)
test/stdlib/requirements/*.py (3 files)

⚠️ Remains on Granite 3.3:

test/backends/test_huggingface.py - See "HuggingFace Test Exception" below

⚠️ Remains on Granite 3.2:

test/backends/test_vision_ollama.py - See "Vision Model Exception" below

Documentation Updates

docs/tutorial.md: Updated all examples to Granite 4
docs/alora.md: Updated training examples, added note about non-hybrid models for adapter training
docs/examples/*.py: Updated all example scripts

Test Infrastructure

Removed 48GB memory markers (Granite 4 micro models require ~16GB)
Fixed CI memory constraints by using MICRO models for Ollama tests
Restored per-backend model selection for IBM_GRANITE_4_MICRO_3B (matches upstream pattern)

HuggingFace Test Exception

HuggingFace tests remain on Granite 3.3 due to missing aLoRA adapters for Granite 4.

The HF tests require the requirement_check intrinsic adapter, which is only available for Granite 3.x models in ibm-granite/rag-intrinsics-lib. While ibm-granite/granite-lib-rag-r1.0 has Granite 4 support for RAG intrinsics (answerability, context_relevance, etc.), the core intrinsics needed for tests are not yet available.

Follow-up Issue: #359 tracks migration once Granite 4 adapters are released.

Vision Model Exception

Vision tests remain on granite3.2-vision due to Ollama compatibility issues.

The ibm/granite3.3-vision:2b model causes Ollama server crashes with segmentation fault (null pointer dereference in llama runner). Reverted to granite3.2-vision which works reliably.

Follow-up Issue: #360 documents the crash with full stack traces and debugging information.

Testing

Tests added to the respective file if code was changed
New code has 100% coverage if code as added
Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Local Testing

# Fast tests (skip LLM quality checks)
uv run pytest -m "not qualitative"

# Full test suite
uv run pytest

Test Results: 204 passed, 6 skipped, 69 deselected, 1 xpassed

CI Testing

All tests pass in CI with CICD=1 (skips qualitative markers).

Related Issues

Closes feat: Update tests & examples to use granite4 #344 (Granite 4 migration)
Follow-up Migrate HuggingFace Tests to Granite 4 When Adapters Available #359 (HuggingFace Granite 4 adapter migration)
Follow-up Investigate granite-vision-3.3-2b Ollama Compatibility Issue #360 (Vision model Ollama crash investigation)

github-actions · 2026-01-26T12:06:01Z

The PR description has been updated. Please fill out the template for your PR to be reviewed.

mergify · 2026-01-26T12:06:25Z

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

planetf1 · 2026-01-26T12:47:23Z

Issue with HuggingFace tests

I moved to using granite v4. First hybrid, then regular. However the intrinsics repo doesn't have granite4 options yet.
Checking status of intrinsics/aLora (including with hybrid - extra parameters for mamba).

planetf1 · 2026-01-26T15:57:37Z

Looking at CI failures...

planetf1 · 2026-01-30T14:35:22Z

Rebased onto upstream/main and squashed to single commit (bc476b9).

Resolved conflicts - added pytest markers from upstream while preserving model selections.

Note: docs/examples/aLora/101_example.py has pre-existing bug (broken since Nov 2025 refactor, commit 1229206). Separate fix needed.

jakelorocco

Looks good! Lets make sure all the intrinsics / adapters tests still run (let me know if you need help testing those). Those tests won't run during the github cicd, so we will need to run them manually.

jakelorocco · 2026-01-27T20:53:35Z

docs/examples/intrinsics/intrinsics.py

 # backend._server_type = _ServerType.REMOTE_VLLM

-backend = LocalHFBackend(model_id="ibm-granite/granite-3.3-8b-instruct")
+backend = LocalHFBackend(model_id="ibm-granite/granite-4.0-tiny-preview")


Note to self: look at this part again.

Just now realized this slipped through. I think this was about whether the intrinsic existed for this model as mentioned below.

jakelorocco · 2026-02-02T14:02:45Z

test/backends/test_openai_vllm/test_openai_vllm.py

Were you able to test if this (and the other) intrinsic / adapter tests still worked? I think in this case at least, there's no requirement_check adapter trained for this model?

I needed #397 to reliably run the tests in a suitable environment (useful when this is merged).

Temporarily I've cherry-picked the commit here & was able to run all the hugging face tests that are active (examples & tests). This is require reverting some of the tests to the granite3.x models -- as some were dependent on adapters that are not yet available. Issue #359 was already open to track that update.

I've also reverted this change. (don't yet have vllm setup - another todo)

GRANITE4_ADAPTER_STATUS.md

planetf1 · 2026-02-03T22:50:47Z

Will look at this tomorrow as I now have a suitable environment

- Update to Granite 4 hybrid models where possible (non-intrinsic tests) - Update watsonx backend to use IBM_GRANITE_4_HYBRID_SMALL as default - Add note in alora.md: use non-hybrid models for adapter training - Remove heavy_ram marker from tests using 3B models (only needed for 8B+) - Update model_ids.py with Granite 4 model mappings and deprecation handling

- Add @pytest.mark.ollama to tests requiring Ollama backend - Update test/README.md with comprehensive marker documentation - Update .gitignore for logs/ and pytest output files

Reverting to match upstream/main to verify if granite-4.0-micro works. Previous commit used granite-3.3-8b-instruct based on assumption that Granite 4 adapters don't exist, but PR generative-computing#397 suggests granite-4.0-micro may work. Testing to confirm.

…am/main) All intrinsics examples were incorrectly changed to granite-3.3-2b-instruct in commit 3b86b9e, but adapters don't exist for the 2B model. Reverting to granite-4.0-micro which has adapters in ibm-granite/granite-lib-rag-r1.0. This matches upstream/main and allows all intrinsics examples to run successfully.

…am/main) The requirement_check adapter only exists for granite-3.3-{2b,8b}-instruct models in ibm-granite/rag-intrinsics-lib, not for granite-4.0-micro. Upstream/main uses granite-3.3-8b-instruct which has the required adapter.

…ain) The requirement_check adapter only exists for granite-3.3-{2b,8b}-instruct models, not for granite-4.0-h-tiny. Upstream/main uses granite-3.3-8b-instruct which has the required adapter in ibm-granite/rag-intrinsics-lib.

planetf1 · 2026-02-04T14:08:11Z

Suggestion: If we can agree/merge #397 I will then rebase this PR, resolve conflicts, and rerun the full suite locally + hugging face remotely.

jakelorocco · 2026-02-04T14:31:59Z

Suggestion: If we can agree/merge #397 I will then rebase this PR, resolve conflicts, and rerun the full suite locally + hugging face remotely.

I've approved it!

planetf1 force-pushed the feat/issue-344 branch from 8c9543e to d19052b Compare January 26, 2026 12:11

planetf1 marked this pull request as ready for review January 26, 2026 15:43

planetf1 force-pushed the feat/issue-344 branch 2 times, most recently from 1e70336 to 014c09d Compare January 28, 2026 08:44

planetf1 requested review from jakelorocco and nrfulton January 29, 2026 12:43

planetf1 force-pushed the feat/issue-344 branch from 014c09d to bc476b9 Compare January 30, 2026 14:35

planetf1 mentioned this pull request Jan 30, 2026

fix: formatting model_ids for better readability #386

Merged

8 tasks

jakelorocco requested changes Feb 2, 2026

View reviewed changes

planetf1 force-pushed the feat/issue-344 branch from bc476b9 to 3b86b9e Compare February 4, 2026 11:48

planetf1 added 5 commits February 4, 2026 13:19

test: add Ollama markers and improve test documentation

c2e176c

- Add @pytest.mark.ollama to tests requiring Ollama backend - Update test/README.md with comprehensive marker documentation - Update .gitignore for logs/ and pytest output files

Conversation

planetf1 commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Migrate from Granite 3.x to Granite 4.0 Models

Type of PR

Description

Summary

Changes

Model Definitions (mellea/backends/model_ids.py)

Backend Updates

Test Updates (19 files)

Documentation Updates

Test Infrastructure

HuggingFace Test Exception

Vision Model Exception

Testing

Local Testing

CI Testing

Related Issues

Uh oh!

github-actions bot commented Jan 26, 2026

Uh oh!

mergify bot commented Jan 26, 2026

Merge Protections

🟢 Enforce conventional commit

Uh oh!

planetf1 commented Jan 26, 2026

Uh oh!

planetf1 commented Jan 26, 2026

Uh oh!

planetf1 commented Jan 30, 2026

Uh oh!

jakelorocco left a comment

Choose a reason for hiding this comment

Uh oh!

jakelorocco Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

jakelorocco Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

jakelorocco Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

planetf1 Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

planetf1 commented Feb 3, 2026

Uh oh!

planetf1 commented Feb 4, 2026

Uh oh!

jakelorocco commented Feb 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

planetf1 commented Jan 26, 2026 •

edited

Loading

Model Definitions (`mellea/backends/model_ids.py`)