小编Gre*_*een的帖子

如何使用awk命令从下表中查找最大值和最小值.

20 90 60 30
55 75 80 85
10 15 99 95
55 95 70 20
9  35 85 75

我想输出像max value=99和min=9

我在系统上安装了 Spark 3.x，并从 Unix 终端以纱线集群模式提交 Spark 命令，并收到以下错误。

(base) vijee@vijee-Lenovo-IdeaPad-S510p:~/spark-3.0.1-bin-hadoop2.7$ bin/spark-submit --master yarn --deploy-mode cluster --py-files /home/vijee/Python/PythonScriptOnYARN.py

并抛出以下错误

Error: Missing application resource 

这是我的查询的完整详细信息：

(base) vijee@vijee-Lenovo-IdeaPad-S510p:~/spark-3.0.1-bin-hadoop2.7$ bin/spark-submit --master yarn --deploy-mode "cluster" --driver-memory 1G --executor-cores 2 --num-executors 1 --executor-memory 2g --py-files /home/vijee/Python/PythonScriptOnYARN.py
Error: Missing application resource.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/vijee/spark-3.0.1-bin-hadoop2.7/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/vijee/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Usage: spark-submit [options] <app jar | python …

我是Apache Pig和Python的新手.当我尝试在Pig中注册Python函数时,它会给出一些与Jython相关的错误.我的python脚本udf1.py将任何字符串转换为大写.

from pig_util import outputSchema

@outputSchema('output_field_name:chararray')
def charupper(x):
    b = x.upper()
    return b

c=charlower('bbbb')

print(c)

当我尝试在Grunt shell中注册Pig本地模式时,它会抛出以下错误

grunt> REGISTER '/home/cloudera/PycharmProjects/Project1/udf1.py' USING jython as pyudf                      
2015-04-06 22:31:45,792 [main] WARN  org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
2015-04-06 22:31:45,793 [main] WARN  org.apache.hadoop.conf.Configuration - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2015-04-06 22:31:45,836 [main] WARN  org.apache.pig.scripting.jython.JythonScriptEngine - pig.cmd.args.remainders is empty. This is not expected unless on testing.
2015-04-06 22:31:45,842 [main] WARN  org.apache.pig.scripting.jython.JythonScriptEngine - module file does not exist: encodings, /usr/lib/pig/lib/jython-standalone-2.5.2.jar/Lib/encodings/__init__.py …

我是PySpark的新手.我用pandas拉了一个csv文件.并使用registerTempTable函数创建了临时表.

from pyspark.sql import SQLContext
from pyspark.sql import Row
import pandas as pd
sqlc = SQLContext(sc)

aa1 = pd.read_csv("D:\mck1.csv")

aa2 = sqlc.createDataFrame(aa1)

aa2.show()

+--------+-------+----------+------------+---------+------------+-------------------+
|    City|     id|First_Name|Phone_Number|new_date|new      code|           New_date|
+--------+-------+----------+------------+---------+------------+-------------------+
|KOLKATTA|9000007|       AAA|  1111119411| 20080714|          13|2016-08-16 00:00:00|
|KOLKATTA|9000007|       BBB|  1111119421| 20080714|          13|2016-08-06 00:00:00|
|KOLKATTA|9000007|       CCC|  1111119461| 20080714|          13|2016-08-13 00:00:00|
|KOLKATTA|9000007|       DDD|  1111119471| 20080714|          13|2016-08-27 00:00:00|
|KOLKATTA|9000007|       EEE|  1111119491| 20080714|          13|2016-08-15 00:00:00|
|KOLKATTA|9111147|       FFF|  1111119401| 20080714|          13|2016-08-24 00:00:00|
|KOLKATTA|9585458|   FORMULA|  1111110112| 19990930|          13|2016-08-16 00:00:00|
|KOLKATTA|9569878|   APPLEII|  1111110132| 19990930|          13|2016-08-06 00:00:00|

aa3 = …

谁能解释一下正则表达式在sed substitute命令中是如何工作的.

$ cat path.txt
/usr/kbos/bin:/usr/local/bin:/usr/jbin:/usr/bin:/usr/sas/bin
/usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin:
/opt/omni/lbin:/opt/omni/sbin:/root/bin

$ sed 's/\(\/[^:]*\).**/\1/g' path.txt
/usr/kbos/bin
/usr/local/sbin
/opt/omni/lbin

从上面的sed命令,他们使用了返回引用和保存运算符的概念.任何人都可以解释一下正则表达式特别是/ [^:]*如何在substitute命令中工作以获得每行中的第一个路径.

我使用的是Python 2.7.3版.当我尝试运行以下代码时,出现错误,

>>> value = input("get value: ")

    get value: hello

    Traceback (most recent call last):

    File "<stdin>", line 1, in <module>

    File "<string>", line 1, in <module>

    NameError: name 'hello' is not defined