小编Vu *_*Anh的帖子

今天我试图从我的分类模型中绘制混淆矩阵.

在一些页面搜索后,我发现matshow从中pyplot可以帮助我.

import matplotlib.pyplot as plt
from sklearn.metrics import confusion_matrix

def plot_confusion_matrix(cm, title='Confusion matrix', cmap=plt.cm.Blues, labels=None):
    fig = plt.figure()
    ax = fig.add_subplot(111)
    cax = ax.matshow(cm)
    plt.title(title)
    fig.colorbar(cax)
    if labels:
        ax.set_xticklabels([''] + labels)
        ax.set_yticklabels([''] + labels)
    plt.xlabel('Predicted')
    plt.ylabel('True')
    plt.show()

如果我的标签很少,它的效果很好

y_true = ['a', 'b', 'c', 'd', 'a', 'b', 'c', 'a', 'c', 'd', 'b', 'a', 'b', 'a']
y_pred = ['a', 'b', 'c', 'd', 'a', 'b', 'b', 'a', 'c', 'a', 'a', 'a', 'a', 'a']
labels = list(set(y_true))
cm = …

我用这个命令将火花应用推向纱线集群

export YARN_CONF_DIR=conf
bin/spark-submit --class "Mining"
  --master yarn-cluster
  --executor-memory 512m ./target/scala-2.10/mining-assembly-0.1.jar

在Web UI中,它坚持下去 UNDEFINED

在控制台,它坚持

<code>14/11/12 16:37:55 INFO yarn.Client: Application report from ASM: 
     application identifier: application_1415704754709_0017
     appId: 17
     clientToAMToken: null
     appDiagnostics: 
     appMasterHost: example.com
     appQueue: default
     appMasterRpcPort: 0
     appStartTime: 1415784586000
     yarnAppState: RUNNING
     distributedFinalState: UNDEFINED
     appTrackingUrl: http://example.com:8088/proxy/application_1415704754709_0017/
     appUser: rain
</code>

更新:

深入了解Logs for containerWeb UI http://example.com:8042/node/containerlogs/container_1415704754709_0017_01_000001/rain/stderr/?start=0,我发现了这一点

14/11/12 02:11:47 WARN YarnClusterScheduler: Initial job has not accepted 
any resources; check your cluster UI to ensure that workers are …