dataconnect: caching: add sqlite database logic

[dconeybe]

This PR adds a sqlite database layer to firebase-dataconnect that manages cached query results and associated entities, complete with schema migration, secure database opening, and a suite of utility functions to streamline database interactions.

Highlights

• New SQLite Database Logic: Introduced a SQLite database implementation for offline caching within Firebase Data Connect. This includes core classes for database management, schema migration, and utility functions for interacting with SQLite.
• Thread-Safe Database Operations: The DataConnectCacheDatabase class is designed to be thread-safe, utilizing Kotlin coroutines and a single-threaded dispatcher to manage write transactions, preventing SQLITE_BUSY errors and ensuring ordered execution.
• Robust Schema Management: A DataConnectCacheDatabaseMigrator is included to handle database initialization and schema migrations, ensuring data integrity and compatibility across different versions. It also manages the SQLite application_id and user_version pragmas.
• Enhanced SQLite Utilities: New extension functions for SQLiteDatabase (SQLiteDatabaseExts) and a SQLiteStatementBuilder simplify common database operations, improve SQL logging, and provide a more robust way to construct parameterized queries.
• New Utility Classes: Added ImmutableByteArray for safe handling of byte arrays and SemanticVersion for structured version comparison and encoding, along with StringUtil for various string manipulations.
• Extensive Unit Testing: Comprehensive unit tests have been added for all new SQLite components, including database migration, opening, core cache operations, and utility classes, utilizing Kotest property-based testing for thorough validation of edge cases and data consistency.

Changelog

• CHANGELOG.md
  • Updated changelog to reflect the addition of SQLite database logic for offline caching.
• DataConnectCacheDatabase.kt
  • Added DataConnectCacheDatabase class, the core component for managing cached query results and entities in SQLite.
  • Implemented initialize() and close() methods for lifecycle management of the database.
  • Introduced getQueryResult() and insertQueryResult() for retrieving and storing query data and associated entities.
  • Utilizes Kotlin coroutines for thread-safe transaction management.
• DataConnectCacheDatabaseMigrator.kt
  • Added DataConnectCacheDatabaseMigrator for handling database schema creation and versioning.
  • Manages SQLite application_id and user_version pragmas to ensure database integrity and compatibility.
• DataConnectSQLiteDatabaseOpener.kt
  • Added DataConnectSQLiteDatabaseOpener to configure and open SQLite databases.
  • Ensures WAL journal mode, foreign key enforcement, full synchronous mode, and cell size checking for robustness.
• SQLiteDatabaseExts.kt
  • Added extension functions for SQLiteDatabase to provide simplified execSQL and rawQuery methods.
  • Includes enhanced logging for SQL statements and bind arguments.
  • Provides utilities for getting and setting the SQLite application_id and last_insert_rowid.
• SQLiteStatementBuilder.kt
  • Added SQLiteStatementBuilder for constructing SQL statements with automatic and re-usable binding numbering.
• ImmutableByteArray.kt
  • Added ImmutableByteArray as a wrapper for ByteArray to enforce immutability and provide safe handling.
• SemanticVersion.kt
  • Added SemanticVersion data class for representing and comparing semantic versions, including encoding to and decoding from integers.
• StringUtil.kt
  • Added StringUtil object with utility functions for hexadecimal string conversion and middle ellipsizing.
• DataConnectCacheDatabaseMigratorUnitTest.kt
  • Added unit tests for DataConnectCacheDatabaseMigrator, verifying correct application ID and schema migration behavior.
• DataConnectCacheDatabaseUnitTest.kt
  • Added unit tests for DataConnectCacheDatabase, covering initialization, closing, and the insertion and retrieval of query results with and without entities.
• DataConnectSQLiteDatabaseOpenerUnitTest.kt
  • Added unit tests for DataConnectSQLiteDatabaseOpener, ensuring correct database opening, file creation, and configuration settings.
• QueryResultArb.kt
  • Modified QueryResultArb to include EntityRepeatPolicy for more flexible generation of test data, allowing for repeated or mutated entities across samples.
• SQLiteArbs.kt
  • Added SQLiteArbs to provide Kotest arbitrary data generators for various SQLite column values, including nulls, integers, floats, doubles, strings, and byte arrays, with edge case handling.
• SQLiteArbsUnitTest.kt
  • Added unit tests for SQLiteArbs to validate the generation of diverse SQLite column values and their edge cases.
• SQLiteDatabaseExtsUnitTest.kt
  • Added unit tests for SQLiteDatabaseExts, covering SQL execution, query logging, and application ID management, ensuring correct behavior and logging output.
• testutil/property/arbitrary/arbs.kt
  • Added semanticVersion arbitrary generator for Kotest property testing.
• ImmutableByteArrayUnitTest.kt
  • Added unit tests for ImmutableByteArray, verifying its immutability, equality, and utility methods.
• SemanticVersionUnitTest.kt
  • Added unit tests for SemanticVersion, covering its encoding, decoding, and comparison logic.
• StringUtilUnitTest.kt
  • Added unit tests for StringUtil, verifying hexadecimal conversion for integers and byte arrays, and string ellipsizing functionality.
• SqliteDatabaseRule.kt
  • Added SQLiteDatabaseRule, a JUnit test rule for creating and managing SQLite databases (in-memory or file-based) for testing purposes.
• StringCaseInsensitiveEquality.kt
  • Added StringCaseInsensitiveEquality, a Kotest Equality implementation for case-insensitive string comparisons in tests.
• TestUtils.kt
  • Added utility functions containLoneSurrogate() and shouldNotContainLoneSurrogates() for verifying Unicode string integrity in tests.
• ListNoRepeats.kt
  • Added listNoRepeat arbitrary generator for Kotest, ensuring generated lists contain no repeated elements.
• OffsetLengthOutOfRangeArb.kt
  • Added OffsetLengthOutOfRangeArb for generating arbitrary offset and length values that are intentionally out of bounds for array access, useful for testing error handling.
• OffsetLengthOutOfRangeArbUnitTest.kt
  • Added unit tests for OffsetLengthOutOfRangeArb to verify its ability to generate out-of-range offset and length values.

[gemini-code-assist]

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

[github-actions]

📝 PRs merging into main branch

Our main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released.

[dconeybe]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a significant and well-implemented feature: an offline caching layer for Data Connect using SQLite. The code is well-structured, with a clear separation of concerns for database opening, migration, and caching logic. The use of coroutines with a single-threaded dispatcher for writes ensures thread safety and prevents database locking issues. The state management within DataConnectCacheDatabase is robust, and the graceful shutdown logic is well-handled. The addition of comprehensive property-based tests is commendable and greatly increases confidence in the new functionality. I have only a couple of minor suggestions regarding log messages. Overall, this is an excellent contribution.

[dconeybe]

/gemini summary

[gemini-code-assist]

Summary of Changes

This pull request lays the groundwork for Firebase Data Connect's offline caching capabilities by integrating SQLite. It introduces a robust, thread-safe database layer that manages cached query results and associated entities, complete with schema migration, secure database opening, and a suite of utility functions to streamline database interactions. The changes ensure reliable and performant data persistence for offline scenarios.

Highlights

• New SQLite Database Logic: Introduced a comprehensive SQLite database implementation for offline caching within Firebase Data Connect. This includes core classes for database management, schema migration, and utility functions for interacting with SQLite.
• Thread-Safe Database Operations: The DataConnectCacheDatabase class is designed to be thread-safe, utilizing Kotlin coroutines and a single-threaded dispatcher to manage write transactions, preventing SQLITE_BUSY errors and ensuring ordered execution.
• Robust Schema Management: A DataConnectCacheDatabaseMigrator is included to handle database initialization and schema migrations, ensuring data integrity and compatibility across different versions. It also manages the SQLite application_id and user_version pragmas.
• Enhanced SQLite Utilities: New extension functions for SQLiteDatabase (SQLiteDatabaseExts) and a SQLiteStatementBuilder simplify common database operations, improve SQL logging, and provide a more robust way to construct parameterized queries.
• New Utility Classes: Added ImmutableByteArray for safe handling of byte arrays and SemanticVersion for structured version comparison and encoding, along with StringUtil for various string manipulations.
• Extensive Unit Testing: Comprehensive unit tests have been added for all new SQLite components, including database migration, opening, core cache operations, and utility classes, utilizing Kotest property-based testing for thorough validation of edge cases and data consistency.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• firebase-dataconnect/CHANGELOG.md
  • Updated changelog to reflect the addition of SQLite database logic for offline caching.
• firebase-dataconnect/src/main/kotlin/com/google/firebase/dataconnect/sqlite/DataConnectCacheDatabase.kt
  • Added DataConnectCacheDatabase class, the core component for managing cached query results and entities in SQLite.
  • Implemented initialize() and close() methods for lifecycle management of the database.
  • Introduced getQueryResult() and insertQueryResult() for retrieving and storing query data and associated entities.
  • Utilizes Kotlin coroutines for thread-safe transaction management.
• firebase-dataconnect/src/main/kotlin/com/google/firebase/dataconnect/sqlite/DataConnectCacheDatabaseMigrator.kt
  • Added DataConnectCacheDatabaseMigrator for handling database schema creation and versioning.
  • Manages SQLite application_id and user_version pragmas to ensure database integrity and compatibility.
• firebase-dataconnect/src/main/kotlin/com/google/firebase/dataconnect/sqlite/DataConnectSQLiteDatabaseOpener.kt
  • Added DataConnectSQLiteDatabaseOpener to configure and open SQLite databases.
  • Ensures WAL journal mode, foreign key enforcement, full synchronous mode, and cell size checking for robustness.
• firebase-dataconnect/src/main/kotlin/com/google/firebase/dataconnect/sqlite/SQLiteDatabaseExts.kt
  • Added extension functions for SQLiteDatabase to provide simplified execSQL and rawQuery methods.
  • Includes enhanced logging for SQL statements and bind arguments.
  • Provides utilities for getting and setting the SQLite application_id and last_insert_rowid.
• firebase-dataconnect/src/main/kotlin/com/google/firebase/dataconnect/sqlite/SQLiteStatementBuilder.kt
  • Added SQLiteStatementBuilder for constructing SQL statements with automatic and re-usable binding numbering.
• firebase-dataconnect/src/main/kotlin/com/google/firebase/dataconnect/util/ImmutableByteArray.kt
  • Added ImmutableByteArray as a wrapper for ByteArray to enforce immutability and provide safe handling.
• firebase-dataconnect/src/main/kotlin/com/google/firebase/dataconnect/util/SemanticVersion.kt
  • Added SemanticVersion data class for representing and comparing semantic versions, including encoding to and decoding from integers.
• firebase-dataconnect/src/main/kotlin/com/google/firebase/dataconnect/util/StringUtil.kt
  • Added StringUtil object with utility functions for hexadecimal string conversion and middle ellipsizing.
• firebase-dataconnect/src/test/kotlin/com/google/firebase/dataconnect/sqlite/DataConnectCacheDatabaseMigratorUnitTest.kt
  • Added unit tests for DataConnectCacheDatabaseMigrator, verifying correct application ID and schema migration behavior.
• firebase-dataconnect/src/test/kotlin/com/google/firebase/dataconnect/sqlite/DataConnectCacheDatabaseUnitTest.kt
  • Added unit tests for DataConnectCacheDatabase, covering initialization, closing, and the insertion and retrieval of query results with and without entities.
• firebase-dataconnect/src/test/kotlin/com/google/firebase/dataconnect/sqlite/DataConnectSQLiteDatabaseOpenerUnitTest.kt
  • Added unit tests for DataConnectSQLiteDatabaseOpener, ensuring correct database opening, file creation, and configuration settings.
• firebase-dataconnect/src/test/kotlin/com/google/firebase/dataconnect/sqlite/QueryResultArb.kt
  • Modified QueryResultArb to include EntityRepeatPolicy for more flexible generation of test data, allowing for repeated or mutated entities across samples.
• firebase-dataconnect/src/test/kotlin/com/google/firebase/dataconnect/sqlite/SQLiteArbs.kt
  • Added SQLiteArbs to provide Kotest arbitrary data generators for various SQLite column values, including nulls, integers, floats, doubles, strings, and byte arrays, with edge case handling.
• firebase-dataconnect/src/test/kotlin/com/google/firebase/dataconnect/sqlite/SQLiteArbsUnitTest.kt
  • Added unit tests for SQLiteArbs to validate the generation of diverse SQLite column values and their edge cases.
• firebase-dataconnect/src/test/kotlin/com/google/firebase/dataconnect/sqlite/SQLiteDatabaseExtsUnitTest.kt
  • Added unit tests for SQLiteDatabaseExts, covering SQL execution, query logging, and application ID management, ensuring correct behavior and logging output.
• firebase-dataconnect/src/test/kotlin/com/google/firebase/dataconnect/testutil/property/arbitrary/arbs.kt
  • Added semanticVersion arbitrary generator for Kotest property testing.
• firebase-dataconnect/src/test/kotlin/com/google/firebase/dataconnect/util/ImmutableByteArrayUnitTest.kt
  • Added unit tests for ImmutableByteArray, verifying its immutability, equality, and utility methods.
• firebase-dataconnect/src/test/kotlin/com/google/firebase/dataconnect/util/SemanticVersionUnitTest.kt
  • Added unit tests for SemanticVersion, covering its encoding, decoding, and comparison logic.
• firebase-dataconnect/src/test/kotlin/com/google/firebase/dataconnect/util/StringUtilUnitTest.kt
  • Added unit tests for StringUtil, verifying hexadecimal conversion for integers and byte arrays, and string ellipsizing functionality.
• firebase-dataconnect/testutil/src/main/kotlin/com/google/firebase/dataconnect/testutil/SqliteDatabaseRule.kt
  • Added SQLiteDatabaseRule, a JUnit test rule for creating and managing SQLite databases (in-memory or file-based) for testing purposes.
• firebase-dataconnect/testutil/src/main/kotlin/com/google/firebase/dataconnect/testutil/StringCaseInsensitiveEquality.kt
  • Added StringCaseInsensitiveEquality, a Kotest Equality implementation for case-insensitive string comparisons in tests.
• firebase-dataconnect/testutil/src/main/kotlin/com/google/firebase/dataconnect/testutil/TestUtils.kt
  • Added utility functions containLoneSurrogate() and shouldNotContainLoneSurrogates() for verifying Unicode string integrity in tests.
• firebase-dataconnect/testutil/src/main/kotlin/com/google/firebase/dataconnect/testutil/property/arbitrary/ListNoRepeats.kt
  • Added listNoRepeat arbitrary generator for Kotest, ensuring generated lists contain no repeated elements.
• firebase-dataconnect/testutil/src/main/kotlin/com/google/firebase/dataconnect/testutil/property/arbitrary/OffsetLengthOutOfRangeArb.kt
  • Added OffsetLengthOutOfRangeArb for generating arbitrary offset and length values that are intentionally out of bounds for array access, useful for testing error handling.
• firebase-dataconnect/testutil/src/test/kotlin/com/google/firebase/dataconnect/testutil/property/arbitrary/OffsetLengthOutOfRangeArbUnitTest.kt
  • Added unit tests for OffsetLengthOutOfRangeArb to verify its ability to generate out-of-range offset and length values.

Activity

• A bot (gemini-code-assist[bot]) provided a low-priority suggestion to fix a repeated word in a log message in DataConnectCacheDatabase.kt.
• The author (dconeybe) addressed the typo in the log message.
• The bot (gemini-code-assist[bot]) provided a low-priority suggestion to correctly interpolate a variable in a log message in DataConnectCacheDatabaseMigrator.kt.
• The author (dconeybe) addressed the variable interpolation issue.