小编Sha*_*mal的帖子

可能重复:
如果cron作业已在运行,如何阻止cron作业执行

我的cron工作可能需要2分钟,或者可能需要5个小时才能完成.我需要确保始终执行此作业.

我的问题是:

如果我将它设置为每分钟执行一次,它会在上一个完成之后启动还是同时运行并弄乱数据库？

是否可以将具有iso-8859-13编码的csv数据转换为UTF-8？

我的旧系统没有UTF-8编码,它只使用iso-8859-13.我需要导入的系统没有iso-8859-13,但有两个UTF-8和UTF-16.如果我尝试使用除以外的编码打开csv文件iso-8859-13,则无法识别某些符号.如果我尝试将此类文件导入新系统,则会出现错误,我编码错误.我只能使用windows-1252它导入它,但随后会导入无法识别的符号.我该怎么做才能将其转换为普通编码,如UTF-8？

signed在C 中意味着什么？我有这张表显示:

这说到signed char 128了+127.128也正整数,所以这怎么能是这样+128来+127？或者做128和+127有不同的含义？我指的是Apress Beginning C这本书.

这是一个非常棒的问题.

我正在尝试了解SparkSQL.我一直在关注这里描述的例子:http: //spark.apache.org/docs/1.0.0/sql-programming-guide.html

在Spark-shell中一切正常,但是当我尝试使用sbt构建批处理版本时,我收到以下错误消息: object sql is not a member of package org.apache.spark

不幸的是,我对sbt很新,所以我不知道如何纠正这个问题.我怀疑我需要包含其他依赖项,但我无法弄清楚如何.

这是我正在尝试编译的代码:

/* TestApp.scala */
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf

case class Record(k: Int, v: String)

object TestApp {
 def main(args: Array[String]) {
   val conf = new SparkConf().setAppName("Simple Application")
   val sc = new SparkContext(conf)
   val sqlContext = new org.apache.spark.sql.SQLContext(sc)
   import sqlContext._
   val data = sc.parallelize(1 to 100000)
   val records = data.map(i => new Record(i, "value = "+i))
   val table = createSchemaRDD(records, Record)
   println(">>> " …

我正在尝试从Spark AWS shell中加载来自Amazon AWS S3存储桶的数据.

我咨询过以下资源:

使用Apache Spark从Amazon S3解析文件

如何从Apache Spark访问s3a://文件？

Hortonworks Spark 1.6和S3

Cloudera的

自定义s3端点

我已经下载并解压缩了Apache Spark 2.2.0.在conf/spark-defaults我有以下(注意我替换access-key和secret-key):

spark.hadoop.fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem
spark.hadoop.fs.s3a.access.key=access-key 
spark.hadoop.fs.s3a.secret.key=secret-key

我已经下载hadoop-aws-2.8.1.jar并aws-java-sdk-1.11.179.jar从mvnrepository,并将它们放置在jars/目录中.然后我启动Spark shell:

bin/spark-shell --jars jars/hadoop-aws-2.8.1.jar,jars/aws-java-sdk-1.11.179.jar

在shell中,以下是我尝试从S3存储桶加载数据的方法:

val p = spark.read.textFile("s3a://sparkcookbook/person")

以下是导致的错误:

java.lang.NoClassDefFoundError: org/apache/hadoop/fs/GlobalStorageStatistics$StorageStatisticsProvider
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:348)
  at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134)
  at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
  at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
  at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2654)
  at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
  at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
  at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
  at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
  at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
  at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)

当我尝试按如下方式启动Spark shell时:

bin/spark-shell --packages …

我有以下df:

+---+----+-----+
|sno|dept|color|
+---+----+-----+
|  1|  fn|  red|
|  2|  fn| blue|
|  3|  fn|green|
+---+----+-----+

如果任何颜色列值是red,那么我应该更新颜色列的所有值red,如下所示:

+---+----+-----+
|sno|dept|color|
+---+----+-----+
|  1|  fn|  red|
|  2|  fn|  red|
|  3|  fn|  red|
+---+----+-----+

我无法理解.请帮忙; 我试过以下代码:

val gp=jdbcDF.filter($"dept".contains("fn"))
     //.withColumn("newone",when($"dept"==="fn","RED").otherwise("NULL"))
    gp.show()
gp.map(
  row=>{
    val row1=row.getAs[String](1)
    var row2=row.getAs[String](2)
    val make=if(row1 =="fn") row2="red"
    Row(row(0),row(1),make)
  }
).collect().foreach(println)

当我尝试通过以下方式杀死Docker守护程序时:

docker kill $(docker ps -q)

我收到以下错误:

Error response from daemon: Cannot kill container: cf5fc4b0e5d1: Cannot kill container cf5fc4b0e5d152a7a89682d8835c40c59e9e0c2c41be4aae330ffeb8093814f2: connection error: desc = "transport: dial unix:///var/run/docker/containerd/docker-containerd.sock: timeout": unknown
Error response from daemon: Cannot kill container: 590fab6b49a2: Cannot kill container 590fab6b49a2e3c832a99074a0679558a9f826d79e94bae7be4ca12c3a019b69: connection error: desc = "transport: dial unix:///var/run/docker/containerd/docker-containerd.sock: timeout": unknown

当我尝试通过以下方式停止Docker守护程序:

docker stop $(docker ps -q)

我收到此错误:

Error response from daemon: cannot stop container: cf5fc4b0e5d1: Cannot kill container cf5fc4b0e5d152a7a89682d8835c40c59e9e0c2c41be4aae330ffeb8093814f2: connection error: desc = "transport: dial unix:///var/run/docker/containerd/docker-containerd.sock: timeout": unknown
Error response from …

尝试循环我的Django模板中的变量时,我收到以下错误.有问题的变量是我的DetailView子类中指定的模型的相关对象:

Type /在/ en/applicants/50771459778 /

'Householdmember'对象不可迭代

这是我的models.py档案:

class Applicant(models.Model):
    user              = models.ForeignKey(User, editable=False)
    bank_card_number  = models.CharField(_('Bank card number'),max_length=50, unique=True)
    site_of_interview = models.IntegerField(_('Site of interview'), choices = SITE_CHOICES, default=TIRANA, blank=False)
    housenumber       = models.CharField(_('House Number'),max_length=8)
    address_line1     = models.CharField(_('Address line 1'),max_length=50)
    address_line2     = models.CharField(_('Apt #'),max_length=50,blank=True) 
    municipality      = models.CharField(_('Municipality/commune'),max_length=25)
    district          = models.CharField(_('District'),max_length=25,blank=True)
    urban             = models.IntegerField(_('Area (urban/rural)'), choices = AREA_CHOICES, blank=False)
    postal            = models.CharField(_('Postal code'),max_length=25,blank=True) 

class Householdmember(models.Model):
    applicant         = models.ForeignKey(Applicant)
    first_name        = models.CharField(_('First name'),max_length=50,blank=False)
    middle_name       = models.CharField(_('Middle name'),max_length=50,blank=True) 
    last_name         = models.CharField(_('Last name'),max_length=50,blank=False)
    national_id …

我想在我的kafka经纪人和zookeeper之间使用SASL.当我启动kafka服务器

KAFKA_OPTS="-Djava.security.auth.login.config=/home/kafka/kafka/config/kafka_server_jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf" \
    ./kafka-server-start.sh ../config/server.properties

我收到以下错误:

INFO TGT refresh thread started. (org.apache.zookeeper.Login)
DEBUG Client principal is "kafkabroker1/kafka.eigenroute.com@EIGENROUTE.COM". (org.apache.zookeeper.Login)
DEBUG Server principal is "krbtgt/EIGENROUTE.COM@EIGENROUTE.COM". (org.apache.zookeeper.Login)
INFO TGT valid starting at:        Sat Dec 16 00:32:52 EST 2017 (org.apache.zookeeper.Login)
INFO TGT expires:                  Sat Dec 16 10:32:52 EST 2017 (org.apache.zookeeper.Login)
INFO TGT refresh sleeping until: Sat Dec 16 08:55:41 EST 2017 (org.apache.zookeeper.Login)
INFO Opening socket connection to server devel-2.sjml.com/173.243.38.81:2181. Will attempt to SASL-authenticate using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn)
DEBUG Closing ZkClient... (org.I0Itec.zkclient.ZkClient) …

假设我有以下DataFrame:

scala> val df1 = Seq("a", "b").toDF("id").withColumn("nums", array(lit(1)))
df1: org.apache.spark.sql.DataFrame = [id: string, nums: array<int>]

scala> df1.show()
+---+----+
| id|nums|
+---+----+
|  a| [1]|
|  b| [1]|
+---+----+

我想在nums列中向数组中添加元素,以便得到如下内容:

+---+-------+
| id|nums   |
+---+-------+
|  a| [1,5] |
|  b| [1,5] |
+---+-------+

有没有办法使用.withColumn()DataFrame 的方法来做到这一点？例如

val df2 = df1.withColumn("nums", append(col("nums"), lit(5))) 

我查看了Spark的API文档,但找不到允许我这样做的任何内容.我可能会一起使用split和concat_ws破解某些东西,但如果有可能的话,我宁愿选择更优雅的解决方案.谢谢.