小编Pau*_*uca的帖子

在容器开始运行之后，Docker似乎正在创建一个桥，然后与我的主机网络发生冲突。这不是默认桥docker0，而是在容器启动后创建的另一个桥。我能够根据旧的用户指南链接https://docs.docker.com/v17.09/engine/userguide/networking/default_network/custom-docker0/配置默认桥，但是，我不知道如何配置另一个网桥，以使其与172.17不冲突。

那么，当前的问题是，当该网桥变为活动状态时，我的容器无法访问主机网络上的其他系统。

有任何想法吗？

码头工人的版本：

Version 18.03.1-ce-mac65 (24312)

这就是创建的桥梁。有时不是172.17，但有时是。

br-f7b50f41d024 Link encap:Ethernet  HWaddr 02:42:7D:1B:05:A3  
      inet addr:172.17.0.1  Bcast:172.17.255.255  Mask:255.255.0.0

我正在尝试使用MRUnit测试Hadoop .mapreduce Avro作业.我收到一个NullPointerException,如下所示.我附上了一部分pom和源代码.任何援助将不胜感激.

谢谢

我得到的错误是:

java.lang.NullPointerException
at org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:73)
at org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:91)
at org.apache.hadoop.mrunit.internal.io.Serialization.copyWithConf(Serialization.java:104)
at org.apache.hadoop.mrunit.TestDriver.copy(TestDriver.java:608)
at org.apache.hadoop.mrunit.MapDriverBase.setInputKey(MapDriverBase.java:64)
at org.apache.hadoop.mrunit.MapDriverBase.setInput(MapDriverBase.java:104)
at org.apache.hadoop.mrunit.MapDriverBase.withInput(MapDriverBase.java:218)
at org.lab41.project.mapreduce.ParseMetadataAsTextIntoAvroTest.testMap(ParseMetadataAsTextIntoAvroTest.java:115)
.....

pom片段:

<dependency>
    <groupId>org.apache.mrunit</groupId>
    <artifactId>mrunit</artifactId>
    <version>0.9.0-incubating</version>
    <classifier>hadoop2</classifier>
    <scope>test</scope>
</dependency>


<avro.version>1.7.4</avro.version>
<hadoop.version>2.0.0-mr1-cdh4.1.3</hadoop.version>

<dependency>
    <groupId>org.apache.avro</groupId>
    <artifactId>avro</artifactId>
    <version>${avro.version}</version>
</dependency>

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-client</artifactId>
    <version>${hadoop.version}</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-core</artifactId>
    <version>${hadoop.version}</version>
    <scope>provided</scope>
 </dependency>
 <dependency>
    <groupId>org.apache.avro</groupId>
    <artifactId>avro-mapred</artifactId>
    <version>${avro.version}</version>
    <classifier>hadoop2</classifier>
 </dependency>

以下是测试的摘录:

import static org.junit.Assert.*;

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.Charset;
import java.nio.charset.CharsetEncoder;
import java.text.ParseException;
import java.text.SimpleDateFormat;

import org.apache.avro.mapred.AvroKey;
import org.apache.avro.hadoop.io.AvroSerialization;
import org.apache.avro.mapred.AvroValue; …

The docs for PubSub state that the max payload after decoding is 10MB. My question is whether or not it is advantageous to compress the payload at the publisher before publishing to increase data throughput?

This especially can be helpful if the payload has a high compression ratio like a json formatted payload.