我在尝试将文件从hdfs读入Spark时遇到错误.文件README.md存在于hdfs中
spark@osboxes hadoop]$ hdfs dfs -ls README.md
16/02/26 00:29:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
-rw-r--r-- 1 spark supergroup 4811 2016-02-25 23:38 README.md
Run Code Online (Sandbox Code Playgroud)
在Spark shell中,我给了
scala> val readme = sc.textFile("hdfs://localhost:9000/README.md")
readme: org.apache.spark.rdd.RDD[String] = MapPartitionsRDD[1] at textFile at <console>:27
scala> readme.count
16/02/26 00:25:26 DEBUG BlockManager: Getting local block broadcast_4
16/02/26 00:25:26 DEBUG BlockManager: Level for block broadcast_4 is StorageLevel(true, true, false, true, 1)
16/02/26 00:25:26 DEBUG BlockManager: Getting block broadcast_4 from memory …Run Code Online (Sandbox Code Playgroud) 我试图运行一个简单的Akka流文件接收器示例但没有成功.我可以创建一个Source,运行Flow然后创建一个文件但ByteString没有写入文件.然而,如果我尝试将流输出打印到控制台,我可以这样做.我在这里错过了什么吗?
import akka.stream._
import akka.stream.scaladsl._
import akka.{ NotUsed, Done}
import akka.actor.ActorSystem
import akka.util.ByteString
import scala.concurrent._
import scala.concurrent.duration._
import java.nio.file.Paths
object First extends App {
val source: Source[Int, NotUsed] = Source ( 1 to 100)
implicit val system = ActorSystem("QuickStart")
implicit val materializer = ActorMaterializer()
// works: prints 1-100
//source.runForeach(println) (materializer)
val factorials = source.scan(BigInt(1))((acc,next) => acc * next)
// there is no content in the Sink (file)
/**val result =
factorials
.map(num => ByteString(s"${num}\n"))
.runWith(FileIO.toPath(Paths.get("factorials.txt")))
**/
def lineSink(fileName: String): Sink[String, …Run Code Online (Sandbox Code Playgroud) 我的所有 brew 命令(更新、配置、docker 等)都在我的 Mac v10.14.2 上抛出以下错误:
brew update
fatal: could not read Username for 'https://github.com': terminal prompts disabled
fatal: could not read Username for 'https://github.com': terminal prompts disabled
Error: homebrew/homebrew-dupes does not exist! Run 'brew untap homebrew/homebrew-dupes'
homebrew/homebrew-x11 does not exist! Run 'brew untap homebrew/homebrew-x11'
Run Code Online (Sandbox Code Playgroud)
我已经尝试了以下选项:
1. brew untap homebrew/homebrew-x11 -> 给出与报告相同的错误。
2. brew untap homebrew/homebrew-dupes -> 给出与报告相同的错误。
3. brew resintall 失败,错误与报告相同Failed during: /usr/local/bin/brew update --force
~/.bash_profile 中的 brew 相关条目:
alias brew='brew update && brew'
关于如何摆脱这个问题的任何建议?