小编use*_*100的帖子

位字段和联合大小

#include<stdio.h>
main()
{
    union d
    {
        unsigned int a:1;
        unsigned int b:3;
        unsigned :0;
        unsigned int d:1;
        unsigned int e:1;
    };
    union d aa;
    aa.b=9;
    printf("d.aa.a=%d d.aa.b=%d",aa.a, aa.b);
system("pause");
}
Run Code Online (Sandbox Code Playgroud)

在此问题中,联合的大小将不同于为联合分配的位字段数。谁能解释这个区别..剩余的内存又如何处理?

c

2
推荐指数
1
解决办法
806
查看次数

Cross join between two large datasets in Spark

I have 2 large datasets. First dataset contains about 130 million entries.
The second dataset contains about 40000 entries. The data is fetched from MySQL tables.

I need to do a cross-join but I am getting

java.sql.SQLException: GC overhead limit exceeded
Run Code Online (Sandbox Code Playgroud)

What is the best optimum technique to do this in Scala?

Following is a snippet of my code:

val df1 = (spark.read.jdbc(jdbcURL,configurationLoader.mysql_table1,"id",100,100000,40, MySqlConnection.getConnectionProperties))
val df2 = (spark.read.jdbc(jdbcURL,configurationLoader.mysql_table2, MySqlConnection.getConnectionProperties))
val df2Cache = df2.repartition(40).cache()
val crossProduct = df1.join(df2Cache)
Run Code Online (Sandbox Code Playgroud)

df1 is the …

scala apache-spark apache-spark-sql

2
推荐指数
1
解决办法
2391
查看次数

标签 统计

apache-spark ×1

apache-spark-sql ×1

c ×1

scala ×1