ProjectSparks执行计划中node 的含义是什么?
我有一个包含以下内容的计划:
+- Project [dm_country#population#6a1ad864-235f-4761-9a6d-0ca2a2b40686#834, dm_country#population#country#839, population#17 AS dm_country#population#population#844]
+- Project [dm_country#population#6a1ad864-235f-4761-9a6d-0ca2a2b40686#834, country#12 AS dm_country#population#country#839, population#17]
+- Project [6a1ad864-235f-4761-9a6d-0ca2a2b40686#22 AS dm_country#population#6a1ad864-235f-4761-9a6d-0ca2a2b40686#834, country#12, population#17]
+- RepartitionByExpression [country#12], 1000
+- Union
:- Project [ind#7 AS 6a1ad864-235f-4761-9a6d-0ca2a2b40686#22, country#12, population#17]
: +- Project [ind#7, country#12, population#2 AS population#17]
: +- Project [ind#7, country#1 AS country#12, population#2]
: +- Project [ind#0 AS ind#7, country#1, population#2]
: +- Relation[ind#0,country#1,population#2] JDBCRelation(schema_dbadmin.t_350) [numPartitions=100]
+- LogicalRDD [ind#45, country#46, population#47]
Run Code Online (Sandbox Code Playgroud) 我有一个使用Akka的系统,它目前通过消息队列处理传入的流数据.当记录到达时,它被处理,mq被激活并且记录被传递以在系统内进一步处理.
现在我想添加对使用DB作为输入的支持.
什么是输入源能够处理数据库的方法(应该以接收器可以处理的速度输入> 100M记录 - 所以我假设反应/ akka-streams?)?