Spring Data MongoDB | 带分页的动态搜索

Kes*_*uwa 5 java mongodb spring-data aggregation-framework spring-data-mongodb

我正在尝试对大量产品进行动态搜索。该对象具有多个属性,包括productNamesubCategoryNamecategoryNamebrandName等。用户可以使用这些属性中的任何一个来搜索产品。顺序是固定的,搜索字符串的首要任务是在productName和 then中找到它subCategoryName,依此类推。

\n

我曾经aggregate实现这一点,然后unionWith连接与其他属性匹配的记录。当作为原始查询触发时,它似乎可以工作,但我们还需要对分页的支持,而我无法使用 Spring Data MongoDB 来实现这一点

\n
db.product.aggregate(\n[\n\xc2\xa0 { $match: { "productName" : { "$regex" : "HYPER", "$options" : "i"}, \n\xc2\xa0 "companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]} }},\n\xc2\xa0 { $unionWith: { coll: "product", pipeline: [{ $match: { "subCategoryName" : { "$regex" : "HYPER", "$options" : "i"},\n\xc2\xa0 "companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]}} }] } },\n\xc2\xa0 { $unionWith: { coll: "product", pipeline: [{ $match: { "categoryName" : { "$regex" : "HYPER", "$options" : "i"}, \n\xc2\xa0 "companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]}} }] } },\n\xc2\xa0 { $unionWith: { coll: "product", pipeline: [{ $match: { "brandName" : { "$regex" : "HYPER", "$options" : "i"},\n\xc2\xa0 "companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]}} }] } },\n]\n)\n
Run Code Online (Sandbox Code Playgroud)\n

此外,只有当我们传递确切名称的子字符串时,此查询才有效。例如,如果我使用 NIVEA BODY LOTION 搜索,将返回NIVEA BODY LOTION EXPRESS HYDRATION 200 ML HYPERmart产品,但如果我使用HYDRATION LOTION搜索,则不会返回任何内容

\n

产品示例:

\n
{\n    "_id" : ObjectId("6278c1c2f2570d6f199435b2"),\n    "companyNo" : 10000009,\n    "categoryName" : "BEAUTY and PERSONAL CARE",\n    "brandName" : "HYPERMART",\n    "productName" : "NIVEA BODY LOTION EXPRESS HYDRATION 200 ML HYPERmart",\n    "productImageUrl" : "https://shop-now-bucket.s3.ap-south-1.amazonaws.com/shop-now-bucket/qa/10000009/product/BEAUTY%20%26%20PERSONAL%20CARE/HYPERMART/NIVEA%20BODY%20LOTION%20EXPRESS%20HYDRATION%20200%20ML/temp1652081080302.jpeg",\n    "compressProductImageUrl" : "https://shop-now-bucket.s3.ap-south-1.amazonaws.com/shop-now-bucket/qa/10000009/product/BEAUTY%20%26%20PERSONAL%20CARE/HYPERMART/NIVEA%20BODY%20LOTION%20EXPRESS%20HYDRATION%20200%20ML/temp1652081080302.jpeg",\n    "productPrice" : 249.0,\n    "status" : "ACTIVE",\n    "subCategoryName" : "BODY LOTION & BODY CREAM",\n    "defaultDiscount" : 0.0,\n    "discount" : 7.0,\n    "description" : "Give your skin fast-absorbing moisturisation and make it noticeably smoother for 48-hours with Nivea Express Hydration Body Lotion. The formula with Sea Minerals and Hydra IQ supplies your skin with moisture all day. The new improved formula contains Deep Moisture Serum to lock in deep moisture leaving you with soft and supple skin.",\n    "afterDiscountPrice" : 231.57,\n    "taxPercentage" : 1.0,\n    "availableQuantity" : NumberLong(100),\n    "packingCharges" : 0.0,\n    "available" : true,\n    "featureProduct" : false,\n    "wholesaleProduct" : false,\n    "rewards" : NumberLong(0),\n    "createAt" : ISODate("2022-05-09T07:24:40.286Z"),\n    "createdBy" : "companyAdmin_@+919146670758shivani.patni@apptware.com",\n    "isBulkUpload" : true,\n    "buyPrice" : 0.0,\n    "privateProduct" : false,\n    "comboProduct" : false,\n    "subscribable" : false,\n    "discountAdded" : false,\n    "_class" : "com.apptmart.product.entity.Product"\n}\n
Run Code Online (Sandbox Code Playgroud)\n

我是 MongoDB 新手。任何参考文献将不胜感激。

\n

11t*_*ion 1

这是我在 Spring Boot 中的工作示例。

https://github.com/ConsciousObserver/MongoAggregationTest

您可以/product使用以下命令调用 REST 服务

http://localhost:8080/products?productName=product&brandName=BRAND1&categoryName=CATEGORY2&subCategoryName=SUB_CATEGORY3&pageNumber=0&pageSize=10
Run Code Online (Sandbox Code Playgroud)

实施支持以下

  1. 文本搜索productName(按单词搜索,需要文本搜索索引)
  2. 精确匹配brandName,categoryNamesubCategoryName
  3. pageNumber使用和进行分页pageSize

所有这些都是使用 Spring Data API 实现的。我通常避免在代码中编写本机查询,因为它们在编译时没有经过验证。

所有类都添加到一个 Java 文件中,这只是一个示例,因此最好将所有内容都放在一个位置。

添加下面的代码,以防 GitHub 存储库出现故障。

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.4</version>
        <relativePath /> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example</groupId>
    <artifactId>MongoAggregationTest</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>MongoAggregationTest</name>
    <description>MongoAggregationTest</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-validation</artifactId>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>
Run Code Online (Sandbox Code Playgroud)

MongoAggregationTestApplication.java

package com.example;

import java.util.ArrayList;
import java.util.List;

import javax.annotation.PostConstruct;
import javax.validation.constraints.Max;
import javax.validation.constraints.Min;

import org.bson.BsonDocument;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.aggregation.Aggregation;
import org.springframework.data.mongodb.core.aggregation.AggregationResults;
import org.springframework.data.mongodb.core.aggregation.LimitOperation;
import org.springframework.data.mongodb.core.aggregation.MatchOperation;
import org.springframework.data.mongodb.core.aggregation.SkipOperation;
import org.springframework.data.mongodb.core.index.TextIndexDefinition;
import org.springframework.data.mongodb.core.index.TextIndexDefinition.TextIndexDefinitionBuilder;
import org.springframework.data.mongodb.core.index.TextIndexed;
import org.springframework.data.mongodb.core.mapping.Document;
import org.springframework.data.mongodb.core.mapping.Field;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.CriteriaDefinition;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.data.mongodb.core.query.TextCriteria;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;

@RequiredArgsConstructor
@SpringBootApplication
@Slf4j
public class MongoAggregationTestApplication {

    public static void main(String[] args) {
        SpringApplication.run(MongoAggregationTestApplication.class, args);
    }

    private final MongoTemplate mongoTemplate;

    @PostConstruct
    void prepareData() {
        boolean collectionExists = mongoTemplate.collectionExists(Product.COLLECTION_NAME);

        log.info("####### product collection exists: {}", collectionExists);

        if (!collectionExists) {
            throw new RuntimeException(
                    String.format("Required collection {%s} does not exist", Product.COLLECTION_NAME));
        }

        //Adding index manually ------------- This is required for text search on productName
        TextIndexDefinition textIndex = new TextIndexDefinitionBuilder().onField("productName", 1F).build();
        mongoTemplate.indexOps(Product.class).ensureIndex(textIndex);

        boolean samplesAlreadyAdded = mongoTemplate
                .exists(new Query().addCriteria(Criteria.where("brandName").exists(true)), Product.class);

        //Uncomment to delete all rows from product collection
        //mongoTemplate.getCollection(Product.COLLECTION_NAME).deleteMany(new BsonDocument());

        if (!samplesAlreadyAdded) {
            for (int i = 1; i <= 5; i++) {
                //adds 3 words in productName
                //product name term1
                String productName = "product name term" + i;

                Product product = new Product(null, "ACTIVE", productName, "BRAND" + i, "CATEGORY" + i,
                        "SUB_CATEGORY" + 1);

                mongoTemplate.save(product);

                log.info("Saving sample product to database: {}", product);
            }
        } else {
            log.info("Skipping sample insertion as they're already in DB");
        }
    }
}

@Slf4j
@RestController
@RequiredArgsConstructor
@Validated
class ProductController {
    private final MongoTemplate mongoTemplate;

    //JSR 303 validations are returning 500 when validation fails, instead of 400. Will look into it later
    /**
     * Invoke using follwing command
     * <p>
     * <code>http://localhost:8080/products?productName=product&brandName=BRAND1&categoryName=CATEGORY2&subCategoryName=SUB_CATEGORY3&pageNumber=0&pageSize=10</code>
     * 
     * @param productName
     * @param brandName
     * @param categoryName
     * @param subCategoryName
     * @param pageNumber
     * @param pageSize
     * @return
     */
    @GetMapping("/products")
    public List<Product> getProducts(@RequestParam String productName, @RequestParam String brandName,
            @RequestParam String categoryName, @RequestParam String subCategoryName,
            @RequestParam @Min(0) int pageNumber, @RequestParam @Min(1) @Max(100) int pageSize) {

        log.info(
                "Request parameters: productName: {}, brandName: {}, categoryName: {}, subCategoryName: {}, pageNumber: {}, pageSize: {}",
                productName, brandName, categoryName, subCategoryName, pageNumber, pageSize);
        //Query Start

        TextCriteria productNameTextCriteria = new TextCriteria().matchingAny(productName).caseSensitive(false);
        TextCriteriaHack textCriteriaHack = new TextCriteriaHack();
        textCriteriaHack.addCriteria(productNameTextCriteria);

        //Needs this hack to combine TextCriteria with Criteria in a single query
        //See TextCriteriaHack for details
        MatchOperation productNameTextMatch = new MatchOperation(textCriteriaHack);

        //Exact match
        Criteria brandNameMatch = Criteria.where("brandName").is(brandName);
        Criteria categoryNameMatch = Criteria.where("categoryName").is(categoryName);
        Criteria subCategoryNameMatch = Criteria.where("subCategoryName").is(subCategoryName);

        MatchOperation orMatch = Aggregation
                .match(new Criteria().orOperator(brandNameMatch, categoryNameMatch, subCategoryNameMatch));

        //Pagination setup
        SkipOperation skip = Aggregation.skip((long) pageNumber * pageSize);
        LimitOperation limit = Aggregation.limit(pageSize);

        Aggregation aggregation = Aggregation.newAggregation(productNameTextMatch, orMatch, skip, limit);

        //Query end

        //Query execution
        AggregationResults<Product> aggregateResults = mongoTemplate.aggregate(aggregation, Product.COLLECTION_NAME,
                Product.class);

        List<Product> products = new ArrayList<>();

        aggregateResults.iterator().forEachRemaining(products::add);

        log.info("Found products: {}", products);

        return products;
    }
}

@Data
@Document(Product.COLLECTION_NAME)
@NoArgsConstructor
@AllArgsConstructor
class Product {
    static final String COLLECTION_NAME = "product";

    @Id
    @Field("_id")
    private String id;

    @Field("status")
    private String status;

    @TextIndexed
    @Field("productName")
    private String productName;

    @Field("brandName")
    private String brandName;

    @Field("categoryName")
    private String categoryName;

    @Field("subCategoryName")
    private String subCategoryName;
}

/**
 * /sf/answers/2094811351/ There is no way to combine
 * CriteriaDefinition and Criteria in one query This hack converts
 * CriteriaDefinition to Query which can be converted to Criteria
 */
class TextCriteriaHack extends Query implements CriteriaDefinition {
    @Override
    public org.bson.Document getCriteriaObject() {
        return this.getQueryObject();
    }

    @Override
    public String getKey() {
        return null;
    }
}
Run Code Online (Sandbox Code Playgroud)

这是正在执行的查询,我从日志/products中获取它MongoTemplate

[
    {
        "$match": {
            "$text": {
                "$search": "name",
                "$caseSensitive": false
            }
        }
    },
    {
        "$match": {
            "$or": [
                {
                    "brandName": "BRAND1"
                },
                {
                    "categoryName": "CATEGORY2"
                },
                {
                    "subCategoryName": "SUB_CATEGORY3"
                }
            ]
        }
    },
    {
        "$skip": 0
    },
    {
        "$limit": 1
    }
]
Run Code Online (Sandbox Code Playgroud)

这是一些请求被触发后的日志内容

2022-10-06 04:50:01.209  INFO 26472 --- [           main] c.e.MongoAggregationTestApplication      : No active profile set, falling back to 1 default profile: "default"
2022-10-06 04:50:01.770  INFO 26472 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data MongoDB repositories in DEFAULT mode.
2022-10-06 04:50:01.780  INFO 26472 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 6 ms. Found 0 MongoDB repository interfaces.
2022-10-06 04:50:02.447  INFO 26472 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2022-10-06 04:50:02.456  INFO 26472 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2022-10-06 04:50:02.456  INFO 26472 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.65]
2022-10-06 04:50:02.531  INFO 26472 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2022-10-06 04:50:02.531  INFO 26472 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1277 ms
2022-10-06 04:50:02.679  INFO 26472 --- [           main] org.mongodb.driver.client                : MongoClient with metadata {"driver": {"name": "mongo-java-driver|sync|spring-boot", "version": "4.6.1"}, "os": {"type": "Windows", "name": "Windows 10", "architecture": "amd64", "version": "10.0"}, "platform": "Java/OpenLogic-OpenJDK/1.8.0-262-b10"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, streamFactoryFactory=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.Jep395RecordCodecProvider@22bd2039]}, clusterSettings={hosts=[localhost:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='30000 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, sendBufferSize=0}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, sendBufferSize=0}, connectionPoolSettings=ConnectionPoolSettings{maxSize=100, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=JAVA_LEGACY, serverApi=null, autoEncryptionSettings=null, contextProvider=null}
2022-10-06 04:50:02.725  INFO 26472 --- [localhost:27017] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:1, serverValue:121}] to localhost:27017
2022-10-06 04:50:02.725  INFO 26472 --- [localhost:27017] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:2, serverValue:122}] to localhost:27017
2022-10-06 04:50:02.726  INFO 26472 --- [localhost:27017] org.mongodb.driver.cluster               : Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=13, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=48972600}
2022-10-06 04:50:02.922  INFO 26472 --- [           main] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:3, serverValue:123}] to localhost:27017
2022-10-06 04:50:02.933  INFO 26472 --- [           main] c.e.MongoAggregationTestApplication      : ####### product collection exists: true
2022-10-06 04:50:02.957 DEBUG 26472 --- [           main] o.s.data.mongodb.core.MongoTemplate      : Executing count: { "brandName" : { "$exists" : true}} in collection: product
2022-10-06 04:50:02.977 DEBUG 26472 --- [           main] o.s.data.mongodb.core.MongoTemplate      : Saving Document containing fields: [status, productName, brandName, categoryName, subCategoryName, _class]
2022-10-06 04:50:02.993  INFO 26472 --- [           main] c.e.MongoAggregationTestApplication      : Saving sample product to database: Product(id=633e1122297cce382aea07d4, status=ACTIVE, productName=product name term1, brandName=BRAND1, categoryName=CATEGORY1, subCategoryName=SUB_CATEGORY1)
2022-10-06 04:50:02.993 DEBUG 26472 --- [           main] o.s.data.mongodb.core.MongoTemplate      : Saving Document containing fields: [status, productName, brandName, categoryName, subCategoryName, _class]
2022-10-06 04:50:02.995  INFO 26472 --- [           main] c.e.MongoAggregationTestApplication      : Saving sample product to database: Product(id=633e1122297cce382aea07d5, status=ACTIVE, productName=product name term2, brandName=BRAND2, categoryName=CATEGORY2, subCategoryName=SUB_CATEGORY1)
2022-10-06 04:50:02.995 DEBUG 26472 --- [           main] o.s.data.mongodb.core.MongoTemplate      : Saving Document containing fields: [status, productName, brandName, categoryName, subCategoryName, _class]
2022-10-06 04:50:02.996  INFO 26472 --- [           main] c.e.MongoAggregationTestApplication      : Saving sample product to database: Product(id=633e1122297cce382aea07d6, status=ACTIVE, productName=product name term3, brandName=BRAND3, categoryName=CATEGORY3, subCategoryName=SUB_CATEGORY1)
2022-10-06 04:50:02.996 DEBUG 26472 --- [           main] o.s.data.mongodb.core.MongoTemplate      : Saving Document containing fields: [status, productName, brandName, categoryName, subCategoryName, _class]
2022-10-06 04:50:02.997  INFO 26472 --- [           main] c.e.MongoAggregationTestApplication      : Saving sample product to database: Product(id=633e1122297cce382aea07d7, status=ACTIVE, productName=product name term4, brandName=BRAND4, categoryName=CATEGORY4, subCategoryName=SUB_CATEGORY1)
2022-10-06 04:50:02.997 DEBUG 26472 --- [           main] o.s.data.mongodb.core.MongoTemplate      : Saving Document containing fields: [status, productName, brandName, categoryName, subCategoryName, _class]
2022-10-06 04:50:02.998  INFO 26472 --- [           main] c.e.MongoAggregationTestApplication      : Saving sample product to database: Product(id=633e1122297cce382aea07d8, status=ACTIVE, productName=product name term5, brandName=BRAND5, categoryName=CATEGORY5, subCategoryName=SUB_CATEGORY1)
2022-10-06 04:50:03.310  INFO 26472 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2022-10-06 04:50:03.318  INFO 26472 --- [           main] c.e.MongoAggregationTestApplication      : Started MongoAggregationTestApplication in 2.446 seconds (JVM running for 2.802)
2022-10-06 04:50:17.447  INFO 26472 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2022-10-06 04:50:17.447  INFO 26472 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2022-10-06 04:50:17.448  INFO 26472 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 1 ms
2022-10-06 04:50:17.511  INFO 26472 --- [nio-8080-exec-1] com.example.ProductController            : Request parameters: productName: product, brandName: BRAND1, categoryName: CATEGORY2, subCategoryName: SUB_CATEGORY3, pageNumber: 0, pageSize: 10
2022-10-06 04:50:17.517 DEBUG 26472 --- [nio-8080