小编tha*_*ndu的帖子

从Kafka主题读取处理数据并使用scala和spark写回Kafka主题

嗨,我正在阅读 kafka 主题,我想处理从 kafka 收到的数据,例如标记化、过滤掉不必要的数据、删除停用词,最后我想写回另一个 Kafka 主题

// read from kafka
val readStream = existingSparkSession
      .readStream
      .format("kafka")
      .option("kafka.bootstrap.servers", hostAddress)
      .option("subscribe", "my.raw") // Always read from offset 0, for dev/testing purpose
      .load()

val df = readStream.selectExpr("CAST(value AS STRING)" )
df.show(false)
val df_json = df.select(from_json(col("value"), mySchema.defineSchema()).alias("parsed_value"))
val df_text = df_json.withColumn("text", col("parsed_value.payload.Text"))

// perform some data processing actions such as tokenization etc and return cleanedDataframe as the final result

// write back to kafka
val writeStream = cleanedDataframe
      .writeStream
      .outputMode("append")
      .format("kafka")
      .option("kafka.bootstrap.servers", hostAddress) …
Run Code Online (Sandbox Code Playgroud)

scala apache-kafka apache-spark spark-structured-streaming

3
推荐指数
1
解决办法
100
查看次数

使用JS的Json数组中相同ID的总和

我有一个ID和年龄如下的json数组

var arrayVal = [{id:"1", age: 20},{id:"2", age: 30},{id:"2", age: "20"},{id:"3", age: 20},{id:"5", age: 10}];
Run Code Online (Sandbox Code Playgroud)

我想获得属于相同ID的年龄总和,如下所示

1 = 20
2 = 50
3 = 20
5 = 10
Run Code Online (Sandbox Code Playgroud)

请找到下面的代码

$scope.TestFunc = function()
{
var tot = 0;
var arrayVal = [{id:"1", age: 20},{id:"2", age: 30},{id:"2", age: "20"},{id:"3", age: 20},{id:"5", age: 10}];
for(var i=0; i <arrayVal.length; i++ )
{
  for(var j=1; j<arrayVal.length - i; j++ )
  {
    if(arrayVal[i].id == arrayVal[j].id)
    {
      tot = arrayVal[i].age.valueOf() + arrayVal[j].age.valueOf();
    }
    else{
      tot = tot …
Run Code Online (Sandbox Code Playgroud)

javascript arrays

1
推荐指数
1
解决办法
84
查看次数

不显示日按钮和周按钮

我正在使用带有 angularjs 的 fullcalendar 来显示事件。目前月视图工作正常。但是用户界面中没有显示日按钮星期按钮。在左侧和右侧只有今天,显示后退和下一个按钮。我无法弄清楚原因。请找到以下代码。

视图.html

<html>
<head>
<link rel="stylesheet" href="../fullcalendar-4.1.0/packages/core/main.css"></script>
<link rel="stylesheet" href="../fullcalendar-4.1.0/packages/daygrid/main.css"></script>
<link rel="stylesheet" href="../fullcalendar-4.1.0/packages/list/main.css"></script>
<link rel="stylesheet" href="../fullcalendar-4.1.0/packages/timegrid/main.css"></script>
<script type="text/javascript" src="../fullcalendar-4.1.0/packages/core/main.js"></script>
    <script type="text/javascript" src="../fullcalendar-4.1.0/packages/daygrid/main.js"></script>
    <script type="text/javascript" src="../fullcalendar-4.1.0/packages/timegrid/main.js"></script>
    <script type="text/javascript" src="../fullcalendar-4.1.0/packages/list/main.js"></script>
</head>
<script>
$scope.event = {events: [{
            title:'test1',
            start: '2019-05-05 08:00',
            end: '2019-05-10 08:00'
          },
          {
            title:'test2',
            start: '2019-05-05 12:00'
        }]};
    var calendarEl = document.getElementById('calendar');

        var calendar = new FullCalendar.Calendar(calendarEl, {
            events: $scope.event.events,
            plugins: [ 'dayGrid','timeGrid','list' ],
            header: {
    left: 'prev,next today',
    center: 'title',
    right: …
Run Code Online (Sandbox Code Playgroud)

fullcalendar angularjs fullcalendar-4

0
推荐指数
1
解决办法
1325
查看次数