Kes*_*uwa 5 java mongodb spring-data aggregation-framework spring-data-mongodb
我正在尝试对大量产品进行动态搜索。该对象具有多个属性,包括productName
、subCategoryName
、categoryName
、brandName
等。用户可以使用这些属性中的任何一个来搜索产品。顺序是固定的,搜索字符串的首要任务是在productName
和 then中找到它subCategoryName
,依此类推。
我曾经aggregate
实现这一点,然后unionWith
连接与其他属性匹配的记录。当作为原始查询触发时,它似乎可以工作,但我们还需要对分页的支持,而我无法使用 Spring Data MongoDB 来实现这一点
db.product.aggregate(\n[\n\xc2\xa0 { $match: { "productName" : { "$regex" : "HYPER", "$options" : "i"}, \n\xc2\xa0 "companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]} }},\n\xc2\xa0 { $unionWith: { coll: "product", pipeline: [{ $match: { "subCategoryName" : { "$regex" : "HYPER", "$options" : "i"},\n\xc2\xa0 "companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]}} }] } },\n\xc2\xa0 { $unionWith: { coll: "product", pipeline: [{ $match: { "categoryName" : { "$regex" : "HYPER", "$options" : "i"}, \n\xc2\xa0 "companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]}} }] } },\n\xc2\xa0 { $unionWith: { coll: "product", pipeline: [{ $match: { "brandName" : { "$regex" : "HYPER", "$options" : "i"},\n\xc2\xa0 "companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]}} }] } },\n]\n)\n
Run Code Online (Sandbox Code Playgroud)\n此外,只有当我们传递确切名称的子字符串时,此查询才有效。例如,如果我使用 NIVEA BODY LOTION 搜索,将返回NIVEA BODY LOTION EXPRESS HYDRATION 200 ML HYPERmart产品,但如果我使用HYDRATION LOTION搜索,则不会返回任何内容
\n产品示例:
\n{\n "_id" : ObjectId("6278c1c2f2570d6f199435b2"),\n "companyNo" : 10000009,\n "categoryName" : "BEAUTY and PERSONAL CARE",\n "brandName" : "HYPERMART",\n "productName" : "NIVEA BODY LOTION EXPRESS HYDRATION 200 ML HYPERmart",\n "productImageUrl" : "https://shop-now-bucket.s3.ap-south-1.amazonaws.com/shop-now-bucket/qa/10000009/product/BEAUTY%20%26%20PERSONAL%20CARE/HYPERMART/NIVEA%20BODY%20LOTION%20EXPRESS%20HYDRATION%20200%20ML/temp1652081080302.jpeg",\n "compressProductImageUrl" : "https://shop-now-bucket.s3.ap-south-1.amazonaws.com/shop-now-bucket/qa/10000009/product/BEAUTY%20%26%20PERSONAL%20CARE/HYPERMART/NIVEA%20BODY%20LOTION%20EXPRESS%20HYDRATION%20200%20ML/temp1652081080302.jpeg",\n "productPrice" : 249.0,\n "status" : "ACTIVE",\n "subCategoryName" : "BODY LOTION & BODY CREAM",\n "defaultDiscount" : 0.0,\n "discount" : 7.0,\n "description" : "Give your skin fast-absorbing moisturisation and make it noticeably smoother for 48-hours with Nivea Express Hydration Body Lotion. The formula with Sea Minerals and Hydra IQ supplies your skin with moisture all day. The new improved formula contains Deep Moisture Serum to lock in deep moisture leaving you with soft and supple skin.",\n "afterDiscountPrice" : 231.57,\n "taxPercentage" : 1.0,\n "availableQuantity" : NumberLong(100),\n "packingCharges" : 0.0,\n "available" : true,\n "featureProduct" : false,\n "wholesaleProduct" : false,\n "rewards" : NumberLong(0),\n "createAt" : ISODate("2022-05-09T07:24:40.286Z"),\n "createdBy" : "companyAdmin_@+919146670758shivani.patni@apptware.com",\n "isBulkUpload" : true,\n "buyPrice" : 0.0,\n "privateProduct" : false,\n "comboProduct" : false,\n "subscribable" : false,\n "discountAdded" : false,\n "_class" : "com.apptmart.product.entity.Product"\n}\n
Run Code Online (Sandbox Code Playgroud)\n我是 MongoDB 新手。任何参考文献将不胜感激。
\n这是我在 Spring Boot 中的工作示例。
https://github.com/ConsciousObserver/MongoAggregationTest
您可以/product
使用以下命令调用 REST 服务
http://localhost:8080/products?productName=product&brandName=BRAND1&categoryName=CATEGORY2&subCategoryName=SUB_CATEGORY3&pageNumber=0&pageSize=10
Run Code Online (Sandbox Code Playgroud)
实施支持以下
productName
(按单词搜索,需要文本搜索索引)brandName
,categoryName
和subCategoryName
pageNumber
使用和进行分页pageSize
所有这些都是使用 Spring Data API 实现的。我通常避免在代码中编写本机查询,因为它们在编译时没有经过验证。
所有类都添加到一个 Java 文件中,这只是一个示例,因此最好将所有内容都放在一个位置。
添加下面的代码,以防 GitHub 存储库出现故障。
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.4</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>MongoAggregationTest</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>MongoAggregationTest</name>
<description>MongoAggregationTest</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</project>
Run Code Online (Sandbox Code Playgroud)
MongoAggregationTestApplication.java
package com.example;
import java.util.ArrayList;
import java.util.List;
import javax.annotation.PostConstruct;
import javax.validation.constraints.Max;
import javax.validation.constraints.Min;
import org.bson.BsonDocument;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.aggregation.Aggregation;
import org.springframework.data.mongodb.core.aggregation.AggregationResults;
import org.springframework.data.mongodb.core.aggregation.LimitOperation;
import org.springframework.data.mongodb.core.aggregation.MatchOperation;
import org.springframework.data.mongodb.core.aggregation.SkipOperation;
import org.springframework.data.mongodb.core.index.TextIndexDefinition;
import org.springframework.data.mongodb.core.index.TextIndexDefinition.TextIndexDefinitionBuilder;
import org.springframework.data.mongodb.core.index.TextIndexed;
import org.springframework.data.mongodb.core.mapping.Document;
import org.springframework.data.mongodb.core.mapping.Field;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.CriteriaDefinition;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.data.mongodb.core.query.TextCriteria;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
@RequiredArgsConstructor
@SpringBootApplication
@Slf4j
public class MongoAggregationTestApplication {
public static void main(String[] args) {
SpringApplication.run(MongoAggregationTestApplication.class, args);
}
private final MongoTemplate mongoTemplate;
@PostConstruct
void prepareData() {
boolean collectionExists = mongoTemplate.collectionExists(Product.COLLECTION_NAME);
log.info("####### product collection exists: {}", collectionExists);
if (!collectionExists) {
throw new RuntimeException(
String.format("Required collection {%s} does not exist", Product.COLLECTION_NAME));
}
//Adding index manually ------------- This is required for text search on productName
TextIndexDefinition textIndex = new TextIndexDefinitionBuilder().onField("productName", 1F).build();
mongoTemplate.indexOps(Product.class).ensureIndex(textIndex);
boolean samplesAlreadyAdded = mongoTemplate
.exists(new Query().addCriteria(Criteria.where("brandName").exists(true)), Product.class);
//Uncomment to delete all rows from product collection
//mongoTemplate.getCollection(Product.COLLECTION_NAME).deleteMany(new BsonDocument());
if (!samplesAlreadyAdded) {
for (int i = 1; i <= 5; i++) {
//adds 3 words in productName
//product name term1
String productName = "product name term" + i;
Product product = new Product(null, "ACTIVE", productName, "BRAND" + i, "CATEGORY" + i,
"SUB_CATEGORY" + 1);
mongoTemplate.save(product);
log.info("Saving sample product to database: {}", product);
}
} else {
log.info("Skipping sample insertion as they're already in DB");
}
}
}
@Slf4j
@RestController
@RequiredArgsConstructor
@Validated
class ProductController {
private final MongoTemplate mongoTemplate;
//JSR 303 validations are returning 500 when validation fails, instead of 400. Will look into it later
/**
* Invoke using follwing command
* <p>
* <code>http://localhost:8080/products?productName=product&brandName=BRAND1&categoryName=CATEGORY2&subCategoryName=SUB_CATEGORY3&pageNumber=0&pageSize=10</code>
*
* @param productName
* @param brandName
* @param categoryName
* @param subCategoryName
* @param pageNumber
* @param pageSize
* @return
*/
@GetMapping("/products")
public List<Product> getProducts(@RequestParam String productName, @RequestParam String brandName,
@RequestParam String categoryName, @RequestParam String subCategoryName,
@RequestParam @Min(0) int pageNumber, @RequestParam @Min(1) @Max(100) int pageSize) {
log.info(
"Request parameters: productName: {}, brandName: {}, categoryName: {}, subCategoryName: {}, pageNumber: {}, pageSize: {}",
productName, brandName, categoryName, subCategoryName, pageNumber, pageSize);
//Query Start
TextCriteria productNameTextCriteria = new TextCriteria().matchingAny(productName).caseSensitive(false);
TextCriteriaHack textCriteriaHack = new TextCriteriaHack();
textCriteriaHack.addCriteria(productNameTextCriteria);
//Needs this hack to combine TextCriteria with Criteria in a single query
//See TextCriteriaHack for details
MatchOperation productNameTextMatch = new MatchOperation(textCriteriaHack);
//Exact match
Criteria brandNameMatch = Criteria.where("brandName").is(brandName);
Criteria categoryNameMatch = Criteria.where("categoryName").is(categoryName);
Criteria subCategoryNameMatch = Criteria.where("subCategoryName").is(subCategoryName);
MatchOperation orMatch = Aggregation
.match(new Criteria().orOperator(brandNameMatch, categoryNameMatch, subCategoryNameMatch));
//Pagination setup
SkipOperation skip = Aggregation.skip((long) pageNumber * pageSize);
LimitOperation limit = Aggregation.limit(pageSize);
Aggregation aggregation = Aggregation.newAggregation(productNameTextMatch, orMatch, skip, limit);
//Query end
//Query execution
AggregationResults<Product> aggregateResults = mongoTemplate.aggregate(aggregation, Product.COLLECTION_NAME,
Product.class);
List<Product> products = new ArrayList<>();
aggregateResults.iterator().forEachRemaining(products::add);
log.info("Found products: {}", products);
return products;
}
}
@Data
@Document(Product.COLLECTION_NAME)
@NoArgsConstructor
@AllArgsConstructor
class Product {
static final String COLLECTION_NAME = "product";
@Id
@Field("_id")
private String id;
@Field("status")
private String status;
@TextIndexed
@Field("productName")
private String productName;
@Field("brandName")
private String brandName;
@Field("categoryName")
private String categoryName;
@Field("subCategoryName")
private String subCategoryName;
}
/**
* /sf/answers/2094811351/ There is no way to combine
* CriteriaDefinition and Criteria in one query This hack converts
* CriteriaDefinition to Query which can be converted to Criteria
*/
class TextCriteriaHack extends Query implements CriteriaDefinition {
@Override
public org.bson.Document getCriteriaObject() {
return this.getQueryObject();
}
@Override
public String getKey() {
return null;
}
}
Run Code Online (Sandbox Code Playgroud)
这是正在执行的查询,我从日志/products
中获取它MongoTemplate
[
{
"$match": {
"$text": {
"$search": "name",
"$caseSensitive": false
}
}
},
{
"$match": {
"$or": [
{
"brandName": "BRAND1"
},
{
"categoryName": "CATEGORY2"
},
{
"subCategoryName": "SUB_CATEGORY3"
}
]
}
},
{
"$skip": 0
},
{
"$limit": 1
}
]
Run Code Online (Sandbox Code Playgroud)
这是一些请求被触发后的日志内容
2022-10-06 04:50:01.209 INFO 26472 --- [ main] c.e.MongoAggregationTestApplication : No active profile set, falling back to 1 default profile: "default"
2022-10-06 04:50:01.770 INFO 26472 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data MongoDB repositories in DEFAULT mode.
2022-10-06 04:50:01.780 INFO 26472 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 6 ms. Found 0 MongoDB repository interfaces.
2022-10-06 04:50:02.447 INFO 26472 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2022-10-06 04:50:02.456 INFO 26472 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2022-10-06 04:50:02.456 INFO 26472 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.65]
2022-10-06 04:50:02.531 INFO 26472 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2022-10-06 04:50:02.531 INFO 26472 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1277 ms
2022-10-06 04:50:02.679 INFO 26472 --- [ main] org.mongodb.driver.client : MongoClient with metadata {"driver": {"name": "mongo-java-driver|sync|spring-boot", "version": "4.6.1"}, "os": {"type": "Windows", "name": "Windows 10", "architecture": "amd64", "version": "10.0"}, "platform": "Java/OpenLogic-OpenJDK/1.8.0-262-b10"} created with settings MongoClientSettings{readPreference=primary, writeConcern=WriteConcern{w=null, wTimeout=null ms, journal=null}, retryWrites=true, retryReads=true, readConcern=ReadConcern{level=null}, credential=null, streamFactoryFactory=null, commandListeners=[], codecRegistry=ProvidersCodecRegistry{codecProviders=[ValueCodecProvider{}, BsonValueCodecProvider{}, DBRefCodecProvider{}, DBObjectCodecProvider{}, DocumentCodecProvider{}, IterableCodecProvider{}, MapCodecProvider{}, GeoJsonCodecProvider{}, GridFSFileCodecProvider{}, Jsr310CodecProvider{}, JsonObjectCodecProvider{}, BsonCodecProvider{}, EnumCodecProvider{}, com.mongodb.Jep395RecordCodecProvider@22bd2039]}, clusterSettings={hosts=[localhost:27017], srvServiceName=mongodb, mode=SINGLE, requiredClusterType=UNKNOWN, requiredReplicaSetName='null', serverSelector='null', clusterListeners='[]', serverSelectionTimeout='30000 ms', localThreshold='30000 ms'}, socketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=0, receiveBufferSize=0, sendBufferSize=0}, heartbeatSocketSettings=SocketSettings{connectTimeoutMS=10000, readTimeoutMS=10000, receiveBufferSize=0, sendBufferSize=0}, connectionPoolSettings=ConnectionPoolSettings{maxSize=100, minSize=0, maxWaitTimeMS=120000, maxConnectionLifeTimeMS=0, maxConnectionIdleTimeMS=0, maintenanceInitialDelayMS=0, maintenanceFrequencyMS=60000, connectionPoolListeners=[], maxConnecting=2}, serverSettings=ServerSettings{heartbeatFrequencyMS=10000, minHeartbeatFrequencyMS=500, serverListeners='[]', serverMonitorListeners='[]'}, sslSettings=SslSettings{enabled=false, invalidHostNameAllowed=false, context=null}, applicationName='null', compressorList=[], uuidRepresentation=JAVA_LEGACY, serverApi=null, autoEncryptionSettings=null, contextProvider=null}
2022-10-06 04:50:02.725 INFO 26472 --- [localhost:27017] org.mongodb.driver.connection : Opened connection [connectionId{localValue:1, serverValue:121}] to localhost:27017
2022-10-06 04:50:02.725 INFO 26472 --- [localhost:27017] org.mongodb.driver.connection : Opened connection [connectionId{localValue:2, serverValue:122}] to localhost:27017
2022-10-06 04:50:02.726 INFO 26472 --- [localhost:27017] org.mongodb.driver.cluster : Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=13, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=48972600}
2022-10-06 04:50:02.922 INFO 26472 --- [ main] org.mongodb.driver.connection : Opened connection [connectionId{localValue:3, serverValue:123}] to localhost:27017
2022-10-06 04:50:02.933 INFO 26472 --- [ main] c.e.MongoAggregationTestApplication : ####### product collection exists: true
2022-10-06 04:50:02.957 DEBUG 26472 --- [ main] o.s.data.mongodb.core.MongoTemplate : Executing count: { "brandName" : { "$exists" : true}} in collection: product
2022-10-06 04:50:02.977 DEBUG 26472 --- [ main] o.s.data.mongodb.core.MongoTemplate : Saving Document containing fields: [status, productName, brandName, categoryName, subCategoryName, _class]
2022-10-06 04:50:02.993 INFO 26472 --- [ main] c.e.MongoAggregationTestApplication : Saving sample product to database: Product(id=633e1122297cce382aea07d4, status=ACTIVE, productName=product name term1, brandName=BRAND1, categoryName=CATEGORY1, subCategoryName=SUB_CATEGORY1)
2022-10-06 04:50:02.993 DEBUG 26472 --- [ main] o.s.data.mongodb.core.MongoTemplate : Saving Document containing fields: [status, productName, brandName, categoryName, subCategoryName, _class]
2022-10-06 04:50:02.995 INFO 26472 --- [ main] c.e.MongoAggregationTestApplication : Saving sample product to database: Product(id=633e1122297cce382aea07d5, status=ACTIVE, productName=product name term2, brandName=BRAND2, categoryName=CATEGORY2, subCategoryName=SUB_CATEGORY1)
2022-10-06 04:50:02.995 DEBUG 26472 --- [ main] o.s.data.mongodb.core.MongoTemplate : Saving Document containing fields: [status, productName, brandName, categoryName, subCategoryName, _class]
2022-10-06 04:50:02.996 INFO 26472 --- [ main] c.e.MongoAggregationTestApplication : Saving sample product to database: Product(id=633e1122297cce382aea07d6, status=ACTIVE, productName=product name term3, brandName=BRAND3, categoryName=CATEGORY3, subCategoryName=SUB_CATEGORY1)
2022-10-06 04:50:02.996 DEBUG 26472 --- [ main] o.s.data.mongodb.core.MongoTemplate : Saving Document containing fields: [status, productName, brandName, categoryName, subCategoryName, _class]
2022-10-06 04:50:02.997 INFO 26472 --- [ main] c.e.MongoAggregationTestApplication : Saving sample product to database: Product(id=633e1122297cce382aea07d7, status=ACTIVE, productName=product name term4, brandName=BRAND4, categoryName=CATEGORY4, subCategoryName=SUB_CATEGORY1)
2022-10-06 04:50:02.997 DEBUG 26472 --- [ main] o.s.data.mongodb.core.MongoTemplate : Saving Document containing fields: [status, productName, brandName, categoryName, subCategoryName, _class]
2022-10-06 04:50:02.998 INFO 26472 --- [ main] c.e.MongoAggregationTestApplication : Saving sample product to database: Product(id=633e1122297cce382aea07d8, status=ACTIVE, productName=product name term5, brandName=BRAND5, categoryName=CATEGORY5, subCategoryName=SUB_CATEGORY1)
2022-10-06 04:50:03.310 INFO 26472 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2022-10-06 04:50:03.318 INFO 26472 --- [ main] c.e.MongoAggregationTestApplication : Started MongoAggregationTestApplication in 2.446 seconds (JVM running for 2.802)
2022-10-06 04:50:17.447 INFO 26472 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2022-10-06 04:50:17.447 INFO 26472 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2022-10-06 04:50:17.448 INFO 26472 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 1 ms
2022-10-06 04:50:17.511 INFO 26472 --- [nio-8080-exec-1] com.example.ProductController : Request parameters: productName: product, brandName: BRAND1, categoryName: CATEGORY2, subCategoryName: SUB_CATEGORY3, pageNumber: 0, pageSize: 10
2022-10-06 04:50:17.517 DEBUG 26472 --- [nio-8080
归档时间: |
|
查看次数: |
2143 次 |
最近记录: |