Implements filter mechanism and improves filter at backend side for AAS Repository by mdanish98 · Pull Request #516 · eclipse-basyx/basyx-java-server-sdk

mdanish98 · 2024-11-05T13:37:19Z

Description of Changes

The current implementation of CrudRepository does filtering InMemory only. However, it should be done on the backend side only based on specific implementation.

This PR adds support for filtering (Pagination/Metamodel) at the backend only. This PR adds some useful, common, and necessary classes as generic so that the Filtering can be applied easily to other repositories and services.

In this PR, the Filter and filtering at the backend feature is defined as a generic way to be used by other modules but the implementation has been done only for the AAS Repository. The reason is that the size of the PR would be way big if all the modules were considered. After this PR other modules implementation will follow.

Affected methods:
getAll*() -> Pagination

Related Issue

#437, #532

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

…ry-crud

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

mateusmolina-iese · 2024-12-10T08:17:26Z

....common/basyx.core/src/main/java/org/eclipse/digitaltwin/basyx/core/BaSyxCrudRepository.java

+ * 
+ * @author danish
+ */
+public interface BaSyxCrudRepository<T> extends CrudRepository<T, String> {


A few notes:

consider using the Spring interface QuerydsIPredicateExecutor.

I think this generic interface may not be necessary. I would consider adding a class to the core backend of the repositories/services/registries themselves instead.

consider renaming this interface (or interfaces, if you think (2) is valid) to backend; the keyword repository is used with different semantics in the context of this project. We decided to use backend for the backend management of the Spring side - which happens to be called repositories. Maybe we can come up with a better naming convention later 😅

mateusmolina-iese · 2024-12-10T08:29:39Z

...core/src/main/java/org/eclipse/digitaltwin/basyx/common/mongocore/MongoDBCrudRepository.java

+ * 
+ * @author danish
+ */
+public class MongoDBCrudRepository<T> extends SimpleMongoRepository<T, String> implements BaSyxCrudRepository<T>  {


Probably not necessary with QuerydslPredicateExecutor. See querydsl-mongodb.

mateusmolina-iese · 2024-12-10T08:38:21Z

...src/main/java/org/eclipse/digitaltwin/basyx/aasrepository/client/ConnectedAasRepository.java

-	public CursorResult<List<AssetAdministrationShell>> getAllAas(PaginationInfo pInfo) {
+	public CursorResult<List<AssetAdministrationShell>> getAllAas(PaginationInfo pInfo, Filter filter) {
 		String encodedCursor = pInfo.getCursor() == null ? null : Base64UrlEncoder.encode(pInfo.getCursor());
 		return repoApi.getAllAssetAdministrationShells(null, null, pInfo.getLimit(), encodedCursor);


It's currently not matching the capabilities of the http controller. The two nulls could be retrieved from the filter object

mateusmolina-iese · 2024-12-10T08:39:21Z

...test/java/org/eclipse/digitaltwin/basyx/aasrepository/client/TestConnectedAasRepository.java

+	@Test
+	@Ignore
+	public void retrieveAllAasWithSpecificAssetIdAndIdShortFiltering() throws Exception {
+		//TODO: Clients doesn't support filtering, remove this when client has the functionality


the internal api supports it. See remark in ConnectedAasRepository.

mateusmolina-iese · 2024-12-10T08:40:26Z

...org/eclipse/digitaltwin/basyx/aasrepository/client/TestAuthorizedConnectedAasRepository.java


 		AasRepository repo = appContext.getBean(AasRepository.class);
-		repo.getAllAas(PaginationInfo.NO_LIMIT).getResult().stream().map(s -> s.getId()).forEach(repo::deleteAas);
+		repo.getAllAas(PaginationInfo.NO_LIMIT, new AasFilter()).getResult().stream().map(s -> s.getId()).forEach(repo::deleteAas);


AasFilter.NO_FILTER or simply passing null?

mateusmolina-iese · 2024-12-10T09:26:23Z

.../eclipse/digitaltwin/basyx/aasrepository/backend/mongodb/TestAasMongoDBFilterResolution.java

+ * 
+ * @author danish
+ */
+public class TestAasMongoDBFilterResolution {


Consider reusing the TestInMemoryAasFilterResolution by extracting a testsuite. The test coverage is considerably different for component with the same function.

mateusmolina-iese · 2024-12-10T09:29:18Z

...repository-core/src/main/java/org/eclipse/digitaltwin/basyx/aasrepository/AasRepository.java

 	 * @return a list of all found Asset Administration Shells
 	 */
-	public CursorResult<List<AssetAdministrationShell>> getAllAas(PaginationInfo pInfo);
+	public CursorResult<List<AssetAdministrationShell>> getAllAas(PaginationInfo pInfo, Filter filter);


Consider overloading this method. There are lot of instances of an empty filter being passed and this measure would reduce quite a bit the amount changes of this PR. Also make sure to document the @param

mateusmolina-iese · 2024-12-10T09:35:50Z

...yx.core/src/main/java/org/eclipse/digitaltwin/basyx/core/pagination/PaginationUtilities.java

+	    return resolveCursor(pRequest, data, Function.identity());
+	}
+
+	public static <T> String resolveCursor(PaginationInfo pRequest, List<T> data, Function<T, String> idResolver) {


I would consider adding these two abstract methods to PagginationSupport, since it already serves as a util class. Also consider leaving this refactoring to another PR. It's not necessarily related to the scope of the current's PR and increases quite a bit the amount of changes (when considering, for example, the changes in aasdiscoveryservice)

mateusmolina-iese · 2024-12-10T09:36:21Z

...rc/main/java/org/eclipse/digitaltwin/basyx/aasdiscoveryservice/backend/CrudAasDiscovery.java

@@ -56,7 +56,7 @@
 */
 public class CrudAasDiscovery implements AasDiscoveryService {


The changes here are not necessarily related to the PR. See remark in common.pagination

mateusmolina-iese · 2024-12-10T09:36:53Z

...est/java/org/eclipse/digitaltwin/basyx/aasdiscoveryservice/backend/CrudAasDiscoveryTest.java

    @Test
    public void getConfiguredAasDiscoveryName(){
-		AasDiscoveryService service = new SimpleAasDiscoveryFactory(null, CONFIGURED_AAS_DISCOVERY_NAME).create();
+		AasDiscoveryService service = new SimpleAasDiscoveryFactory(() -> null, CONFIGURED_AAS_DISCOVERY_NAME).create();


not related to pr

mateusmolina-iese · 2024-12-10T09:39:42Z

basyx.common/basyx.core/src/main/java/org/eclipse/digitaltwin/basyx/core/Filter.java

@@ -0,0 +1,9 @@
+package org.eclipse.digitaltwin.basyx.core;


License header

aaronzi · 2025-02-20T07:49:56Z

Superseded by implementations related to #584. Keeping this as a draft for now as a reference.

mdanish98 added 8 commits October 16, 2024 17:01

Moves InMemory CrudRepository to inmemory core module

6a51b5c

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

Refactors *InMemory Crud Repo to common

47c8c18

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

Merge remote-tracking branch 'upstream/main' into fix/refactor-inmemo…

9686a99

…ry-crud

Removes NO_LIMIT_PAGINATION_INFO

f8ed64b

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

Implements feature for filtering elements at backend only

1dd8b8f

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

Resolves conflicts

b9b14c5

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

Resets CI files

68217b9

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

Refactors codes and add license headers

c231307

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

mdanish98 changed the title ~~Implements filter mechanism and improves filter at backend side~~ Implements filter mechanism and improves filter at backend side for AAS Repository Nov 5, 2024

mdanish98 added 2 commits November 5, 2024 14:41

Fixes CI

06c9bbc

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

Adds missing import

c404266

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

mdanish98 marked this pull request as ready for review November 5, 2024 14:00

Adds Javadoc and license headers to PaginationUtilities

2c7776e

Signed-off-by: Mohammad Ghazanfar Ali Danish <ghazanfar.danish@iese.fraunhofer.de>

zhangzai123 mentioned this pull request Nov 21, 2024

[BUG] pagination does not improve response time #532

Closed

mateusmolina-iese self-requested a review December 5, 2024 12:16

mateusmolina-iese suggested changes Dec 10, 2024

View reviewed changes

mateusmolina-iese reviewed Dec 10, 2024

View reviewed changes

mdanish98 mentioned this pull request Dec 24, 2024

Support for filtering authorized models based on the defined RBAC rules. #575

Merged

aaronzi mentioned this pull request Jan 10, 2025

Refactor Backend Architecture #584

Closed

1 task

aaronzi marked this pull request as draft February 20, 2025 07:50

aaronzi mentioned this pull request Feb 20, 2025

Filter options for AAS/SM/CD with Authorization #436

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implements filter mechanism and improves filter at backend side for AAS Repository#516

Implements filter mechanism and improves filter at backend side for AAS Repository#516
mdanish98 wants to merge 11 commits intoeclipse-basyx:mainfrom
mdanish98:feature/filter-at-backend

mdanish98 commented Nov 5, 2024 •

edited by zhangzai123

Loading

Uh oh!

mateusmolina-iese Dec 10, 2024

Uh oh!

mateusmolina-iese Dec 10, 2024

Uh oh!

mateusmolina-iese Dec 10, 2024

Uh oh!

mateusmolina-iese Dec 10, 2024

Uh oh!

mateusmolina-iese Dec 10, 2024

Uh oh!

mateusmolina-iese Dec 10, 2024

Uh oh!

mateusmolina-iese Dec 10, 2024 •

edited

Loading

Uh oh!

mateusmolina-iese Dec 10, 2024

Uh oh!

mateusmolina-iese Dec 10, 2024

Uh oh!

mateusmolina-iese Dec 10, 2024

Uh oh!

mateusmolina-iese Dec 10, 2024

Uh oh!

aaronzi commented Feb 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -56,7 +56,7 @@
		*/
		public class CrudAasDiscovery implements AasDiscoveryService {

Conversation

mdanish98 commented Nov 5, 2024 • edited by zhangzai123 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of Changes

Related Issue

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mateusmolina-iese Dec 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aaronzi commented Feb 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mdanish98 commented Nov 5, 2024 •

edited by zhangzai123

Loading

mateusmolina-iese Dec 10, 2024 •

edited

Loading