小编slo*_*ype的帖子

加载sidekiq队列时出现以下错误:

RuntimeError: Circular dependency detected while autoloading constant FileProcessor
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:492:in `load_missing_constant'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:184:in `const_missing'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:526:in `load_missing_constant'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/activesupport-4.2.0/lib/active_support/dependencies.rb:184:in `const_missing'
/Users/johnmcauley/workspace/wripl-capture/lib/wripl_article.rb:23:in `build_from_crawled_page'
/Users/johnmcauley/workspace/wripl-capture/app/workers/article_worker.rb:7:in `perform'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/processor.rb:75:in `execute_job'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/processor.rb:52:in `block (2 levels) in process'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/middleware/chain.rb:127:in `block in invoke'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/middleware/server/active_record.rb:6:in `call'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/middleware/server/retry_jobs.rb:74:in `call'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-failures-0.4.4/lib/sidekiq/failures/middleware.rb:9:in `call'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/middleware/server/logging.rb:15:in `block in call'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/logging.rb:30:in `with_context'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/middleware/server/logging.rb:11:in `call'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/middleware/chain.rb:132:in `call'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/middleware/chain.rb:132:in `invoke'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/processor.rb:51:in `block in process'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/processor.rb:98:in `stats'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/sidekiq-3.4.1/lib/sidekiq/processor.rb:50:in `process'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `public_send'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `dispatch'
/Users/johnmcauley/.rvm/gems/ruby-2.2.2/gems/celluloid-0.16.0/lib/celluloid/calls.rb:122:in …

我正在使用ffmpeg处理一些视频,然后使用aws_s3 gem将视频发送到S3 .我使用以下代码:

S3Object.store("testme.mp4", open(file), 'blah', :access => :public_read)  

一切都很好但有1GB及以上的文件我收到以下错误:

"Timeout::Error: execution expired".  

这只发生在ffmpeg处理完文件之后.如果我自己发送文件而不进行处理,那么一切都很好.

有没有人遇到类似的问题？

谢谢,

SLothistype

我不敢相信我问这个但是......

你如何使用SCALA逃避SPARK SQL中的SQL QUERY STRING？

我厌倦了一切,到处搜寻.我以为apache commons库会这样做,但没有运气:

import org.apache.commons.lang.StringEscapeUtils

var sql = StringEscapeUtils.escapeSql("'Ulmus_minor_'Toledo'");

df.filter("topic = '" + sql + "'").map(_.getValuesMap[Any](List("hits","date"))).collect().foreach(println);

Run Code Online (Sandbox Code Playgroud)

返回以下内容:

topic =''''Ulmus_minor _''Toledo'''^ scala.sys.package $ .error(package.scala:27)at org.apache.spark.sql.catalyst.SqlParser.parseExpression(SqlParser.scala:45) at org.apache.spark.sql.DataFrame.filter(DataFrame.scala:651)at $ iwC $$ iwC $$ iwC $$ iwC $$ iwC $$ iwC $$ iwC $$ iwC $$ iwC.(:29 )$ iwC $$ iwC $$ iwC $$ iwC $$ iwC $$ iwC $$ iwC $$ iwC.(:34)at $ iwC $$ iwC $$ iwC $$ iwC $$ iwC $$ iwC $$ iwC.(:36)at $ iwC …

我使用 sidekiq 3.4.2 和 redis 3.0.3 来执行一些长时间运行的作业，大约两到四个小时后，我收到以下错误：

错误：心跳：EXECABORT 由于先前的错误，事务已被丢弃。

EXECABORT 由于先前的错误而放弃事务。/home/deployer/apps/wripl-capture/shared/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/pipeline.rb:79 finish' /home/deployer/apps/wripl-capture/shared/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:149:in :在 call_pipeline' /home/deployer/apps 中的块中/wripl-capture/shared/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:279:in with_reconnect' /home/deployer/apps/wripl-capture/shared/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis/client.rb:147:in call_pipeline'/home/deployer/apps/wripl-capture/shared/bundle /ruby/2.2.0/gems/redis-3.2.1/lib/redis.rb:2182 block in multi' /home/deployer/apps/wripl-capture/shared/bundle/ruby/2.2.0/gems/redis-3.2.1/lib/redis.rb:37:in :在同步块中'/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0 /monitor.rb:211:在`mon_synchronize'中

除非我重新启动 sidekiq，否则一切都会被终止（没有运行排队的作业），这不是我需要做的。

知道发生了什么吗？