小编sca*_*log的帖子

我改变了我的database.yml,在测试和开发中使用sqlite3数据库,在生产中使用postgresql.我的应用程序在生产中运行良好,但是当我启动测试或开发环境时,我有这样的错误:

Cannot load 'Rails.application.database_configuration':
undefined method'[]' for nil:NilClass (NoMethoError)

我的database.yml:

# SQLite version 3.x
#   gem install sqlite3
#
#   Ensure the SQLite 3 gem is defined in your Gemfile
#   gem 'sqlite3'
#
default: &default
  adapter: sqlite3
  pool: 5
  timeout: 5000

development:
  <<: *default
  database: db/development.sqlite3

# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or …

我构建了一个spark应用程序来计算文件中的单词数.我在cloudera quickstart VM上运行应用程序,当我使用cloudera用户目录时,一切都很好,但是当我想在其他用户目录中写入或读取时,我有权从hadoop拒绝.我想知道如何在spark中更改hadoop用户.

package user1.item1

import user1.{Article}
import org.apache.spark.{SparkContext, SparkConf}
import org.apache.spark.SparkContext._
import scala.util.{Try, Success, Failure}

object WordCount {
  def main(args: Array[String]) {
    Context.User = 'espacechange'
    val filename = "hdfs://quickstart.cloudera:8020/user/user1/test/wiki_test/wikipedia.txt"
    val conf = new SparkConf().setAppName("word count")
    val sc = new SparkContext(conf)
    val wikipedia = sc.textFile(filename).map(Article.parseWikipediaArticle)
    val counts = wikipedia.flatMap(line => line.text.split(" ")).map(word => (word, 1)).reduceByKey(_ + _)

    counts.saveAsTextFile("hdfs://quickstart.cloudera:8020/user/user1/test/word_count")
  }
}