我有几张包含大量数据的表(大约1亿条记录).所以我不能将这些数据存储在内存中,但是我希望使用类来传输这个结果集java.util.stream
并将此流传递给另一个类.我读了关于Stream.of
和Stream.Builder
运算符,但它们是内存中的缓冲流.那么有什么方法可以解决这个问题吗?提前致谢.
更新#1
好吧,我用Google搜索并找到了jooq库.我不确定,但看起来它可能适用于我的测试用例.总而言之,我有几张包含大量数据的表格.我想流式传输我的结果集并将此流传输到另一个方法.像这样的东西:
// why return Stream<String>? Because my result set has String type
private Stream<Record> writeTableToStream(DataSource dataSource, String table) {
Stream<Record> record = null;
try (Connection connection = dataSource.getConnection()) {
String sql = "select * from " + table;
try (PreparedStatement pSt = connection.prepareStatement(sql)) {
connection.setAutoCommit(false);
pSt.setFetchSize(5000);
ResultSet resultSet = pSt.executeQuery();
//
record = DSL.using(connection)
.fetch(resultSet).stream();
}
} catch (SQLException sqlEx) {
logger.error(sqlEx);
}
return record;
}
Run Code Online (Sandbox Code Playgroud)
可以请有人建议,我是否正确?谢谢.
更新#2
我在 …
我有下一堂课:
映射器
public interface DeviceTokensMapper {
DeviceTokensMapper INSTANCE = Mappers.getMapper(DeviceTokensMapper.class);
@Mappings({
@Mapping(source = "tokenName", target = "tokenName"),
@Mapping(source = "userOsType", target = "osType"),
})
DeviceTokensDTO toDeviceTokensDTO(DeviceTokens deviceTokens);
}
Run Code Online (Sandbox Code Playgroud)
实体:
@Entity
public class DeviceTokens {
@Id
@Column(name="token_name", nullable = false)
private String tokenName;
@Column(name = "os", nullable = false)
@Enumerated
private UserOSType userOsType;
public DeviceTokens() {}
public DeviceTokens(String tokenName, UserOSType userOSType) {
this.tokenName = tokenName;
this.userOsType = userOSType;
}
public String getTokenName() {
return tokenName;
}
public void setTokenName(String tokenName) { …
Run Code Online (Sandbox Code Playgroud) 我有下一个xml文件:
<AutoTest>
<Source>EBS FX</Source>
<CreateCFF>No</CreateCFF>
<FoXML descriptor="pb.fx.spotfwd.trade.feed" version="2.0">
<FxSpotFwdTradeFeed>
<FxSpotFwd feed_datetime="17-Dec-2014 10:20:09"
cpty_sds_id="EBS" match_id="L845586141217" original_trade_id_feed="L80107141217"
value_date="20141218" trade_id_external="001-002141880445/5862" match_sds_id="EBSFeedCpty"
counter_ccy="USD" trade_id_feed="107" trade_type="S" feed_source_id="80" quoting_term="M"
deal_ccy="GBP" rate="1.5" trade_date="20141217" modified_by="automation" cpty_side="B" counter_amt="1500000"
smart_match="0" booking_status_id="10" trade_status_id="22" deal_amt="1000000" trade_direction="B">
<Notes />
</FxSpotFwd>
</FxSpotFwdTradeFeed>
<TestCases />
</FoXML>
</AutoTest>
Run Code Online (Sandbox Code Playgroud)
如何使用sed 获取trade_id_external属性的值?
我尝试了这个表达方式:sed -n '/trade_id_external/s/.*=//p' ./file.xml
但没有运气
我尝试在Spring Data项目中配置OneToMany和ManyToOne映射但有一些问题.
所以我有两个实体:雇主和项目.一个雇主可以有很多项目.
实体类:
Employer.java
@Entity
@Table (name="Employer")
public class Employer {
@Id
@SequenceGenerator(name="my_seq", sequenceName="GLOBAL_SEQUENCE")
@GeneratedValue(strategy = GenerationType.SEQUENCE ,generator="my_seq")
@Column (name="employer_id")
private int id;
@Column (name="name")
private String name;
@JoinColumn (name="employer_id")
@OneToMany(mappedBy = "employer", cascade = CascadeType.ALL, fetch = FetchType.EAGER)
private Set<Project> project = new HashSet<Project>();
......................
}
Run Code Online (Sandbox Code Playgroud)
Project.java
@Entity
@Table (name="Project")
public class Project {
@Id
@SequenceGenerator(name="my_seq", sequenceName="GLOBAL_SEQUENCE")
@GeneratedValue(strategy = GenerationType.SEQUENCE ,generator="my_seq")
@Column (name="project_id")
private int id;
@Column (name="name")
private String …
Run Code Online (Sandbox Code Playgroud) 让我们想象我有下一堂课:
public class Service {
public void transferMoney(Account fromAcct, Account toAcct, int amount) {
synchronized (fromAcct) {
synchronized (toAccount) { // could we use here only one synchronized block?
fromAcct.credit(amount);
toAccount.debit(amount);
}
}
}
}
class Account {
private int amount = 0;
public void credit(int sum) {
amount = amount + sum;
}
public void debit(int sum) {
amount = amount - sum;
}
}
Run Code Online (Sandbox Code Playgroud)
例如,我知道我们只能在方法中改变状态fromAcct
和toAcct
对象transferMoney
.那么我们可以用一个同步块重写我们的方法吗?
public class Service {
private …
Run Code Online (Sandbox Code Playgroud) 我有下一个spring rest控制器来处理我的异常:
@ExceptionHandler(ConstraintViolationException.class)
@ResponseStatus(HttpStatus.BAD_REQUEST)
public @ResponseBody
ExceptionResponseDTO hiberAnnotationExceptionHandler(ConstraintViolationException exception) {
return new ExceptionResponseDTO("Entity is null or we are out of range!", exception.getMessage(), ExceptionMapper.INSTANCE.exceptionToExceptionDTO(exception));
}
Run Code Online (Sandbox Code Playgroud)
我的实体类:
@Entity
public class Device {
@Id
@Column(name="name", nullable = false)
private String Name;
@Column(name = "send_date")
@NotNull
private Date sendDate;
}
Run Code Online (Sandbox Code Playgroud)
我尝试模拟ConstraintViolationException,因此在我的控制器中使用下一个代码:
Device d = new Device();
d.setName("sdsdsd");
d.setSendDate(null);
deviceRepository.save(d);
Run Code Online (Sandbox Code Playgroud)
结果,我收到下一个异常:
[dispatcherServlet] :?-具有路径[]的上下文中Servlet [dispatcherServlet]的Servlet.service()引发异常[请求处理失败;嵌套的异常是org.springframework.transaction.TransactionSystemException:无法提交JPA事务。嵌套的异常是javax.persistence.RollbackException:根本原因提交事务时出错,原因是javax.validation.ConstraintViolationException:组[javax.validation.groups.Default,]的更新期间,对类[com.entity.Device]的验证失败违反约束的列表:[ConstraintViolationImpl {interpolatedMessage ='可能不为空',propertyPath = sendDate,rootBeanClass = class com.entity.Device,messageTemplate ='{javax.validation.constraints.NotNull.message}'}]
因此,从堆栈跟踪中可以看到,我首先收到TransactionSystemException,因此,我的ExceptionHandler方法(hiberAnnotationExceptionHandler)不被调用。所以我的问题是如何模拟此异常( …
我想使用 QueryDSL 库构建选择计数查询,如下所示:
select count(1) from table1
我创建了下一个代码:
SQLTemplates sqlTemplates = builder.printSchema().build();
Configuration configuration = new Configuration(sqlTemplates);
Path table = new RelationalPathBase(Object.class, "alias", "hr", "table1");
StringPath[] columnsPath = new StringPath[1];
columnsPath[0] = Expressions.stringPath(table, "field1");
SQLQuery sqlQuery = new SQLQuery(dataSource.getConnection(), configuration)
.from(table);
sqlQuery.setUseLiterals(true);
String selectStatement = sqlQuery.getSQL(columnsPath).getSQL();
Run Code Online (Sandbox Code Playgroud)
结果 selectStatement 是下一个:
select alias.field1 from hr.table1 alias
Run Code Online (Sandbox Code Playgroud)
请有人建议如何重写上面的代码以获得
select count(alias.field1) from hr.table1 alias
Run Code Online (Sandbox Code Playgroud) 我有一个方法,接受块的文件和大小,并返回分块文件的列表.但是我的文件行可能会被破坏的主要问题,例如在主文件中我有下一行:
|1|aaa|bbb|ccc|
|2|ggg|ddd|eee|
Run Code Online (Sandbox Code Playgroud)
拆分后我可以在一个文件中:
|1|aaa|bbb
Run Code Online (Sandbox Code Playgroud)
在另一个文件中:
|ccc|2|
|ggg|ddd|eee|
Run Code Online (Sandbox Code Playgroud)
这是代码:
public static List<File> splitFile(File file, int sizeOfFileInMB) throws IOException {
int counter = 1;
List<File> files = new ArrayList<>();
int sizeOfChunk = 1024 * 1024 * sizeOfFileInMB;
byte[] buffer = new byte[sizeOfChunk];
try (BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file))) {
String name = file.getName();
int tmp = 0;
while ((tmp = bis.read(buffer)) > 0) {
File newFile = new File(file.getParent(), name + "."
+ String.format("%03d", counter++));
try (FileOutputStream out = new …
Run Code Online (Sandbox Code Playgroud) 假设我有下一堂课:
public class Singleton{
private static Singleton _instance;
public static Singleton getInstance(){
if(_instance == null){
synchronized(Singleton.class){
if(_instance == null)
_instance = new Singleton();
}
}
return _instance;
}
Run Code Online (Sandbox Code Playgroud)
例如,我们有两个线程A和B,它们尝试同时执行getInstance()方法。我是否正确理解该过程:
我在执行 MongoOperations 类的 findOne 方法时遇到了一些问题,现在这个方法返回 null。我在 mongoDB 中的数据结构如下所示:
> db.news.find({_id:1})
{ "_id" : 1, "title" : "first title", "text" : "first text" }
> db.news.find({_id:{$type:1}})
{ "_id" : 1, "title" : "first title", "text" : "first text" }
Run Code Online (Sandbox Code Playgroud)
正如您在上面看到的 _id 字段具有 Double 类型。我的 Java 类看起来像这样:
@Repository
public class NewsService {
@Autowired
private MongoOperations mongoOperations;
public static final String COLLECTION_NAME = "news";
//this method executes ok
public List<NewsEntity> getAllNews() {
return mongoOperations.findAll(NewsEntity.class, COLLECTION_NAME);
}
//but this method return null
public NewsEntity …
Run Code Online (Sandbox Code Playgroud) java ×8
spring ×3
hibernate ×2
bash ×1
concurrency ×1
java-stream ×1
jdbc ×1
jooq ×1
lambda ×1
mapstruct ×1
mongodb ×1
querydsl ×1
regex ×1
sed ×1
spring-mvc ×1
synchronized ×1