小编Him*_*oon的帖子

val columnName=Seq("col1","col2",....."coln");

有没有办法执行dataframe.select操作以获取仅包含指定列名的数据帧.我知道我可以做,dataframe.select("col1","col2"...) 但是columnName在运行时生成.我可以dataframe.select()在循环中为每个列名重复执行.它会有任何性能开销吗？有没有其他更简单的方法来实现这一目标？

在tensorflow 2.0 指南中的tf 2.0 DC Gan 示例中，有两个梯度磁带。见下文。

@tf.function
def train_step(images):
    noise = tf.random.normal([BATCH_SIZE, noise_dim])

    with tf.GradientTape() as gen_tape, tf.GradientTape() as disc_tape:
      generated_images = generator(noise, training=True)

      real_output = discriminator(images, training=True)
      fake_output = discriminator(generated_images, training=True)

      gen_loss = generator_loss(fake_output)
      disc_loss = discriminator_loss(real_output, fake_output)

    gradients_of_generator = gen_tape.gradient(gen_loss, generator.trainable_variables)
    gradients_of_discriminator = disc_tape.gradient(disc_loss, discriminator.trainable_variables)

    generator_optimizer.apply_gradients(zip(gradients_of_generator, generator.trainable_variables))
    discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator, discriminator.trainable_variables))

如您所见，有两个渐变色带。我想知道使用单个磁带有什么区别并将其更改为以下内容

@tf.function
def train_step(images):
    noise = tf.random.normal([BATCH_SIZE, noise_dim])

    with tf.GradientTape() as tape:
      generated_images = generator(noise, training=True)

      real_output = discriminator(images, training=True)
      fake_output = discriminator(generated_images, training=True)

      gen_loss = …

在 TensorFlow 1.X 中，您可以使用占位符动态更改批次大小。例如

dataset.batch(batch_size=tf.placeholder())
查看完整示例

你如何在 TensorFlow 2.0 中做到这一点？

我已经尝试了以下但它不起作用。

import numpy as np
import tensorflow as tf


def new_gen_function():
    for i in range(100):
        yield np.ones(2).astype(np.float32)


batch_size = tf.Variable(5, trainable=False, dtype=tf.int64)
train_ds = tf.data.Dataset.from_generator(new_gen_function, output_types=(tf.float32)).batch(
    batch_size=batch_size)

for data in train_ds:
    print(data.shape[0])
    batch_size.assign(10)
    print(batch_size)

输出

5
<tf.Variable 'Variable:0' shape=() dtype=int64, numpy=10>
5
<tf.Variable 'Variable:0' shape=() dtype=int64, numpy=10>
5
<tf.Variable 'Variable:0' shape=() dtype=int64, numpy=10>
5
...
...

我正在使用 Gradient 磁带使用自定义训练循环训练模型。我怎样才能做到这一点？

我正在尝试在TensorFlow 2.0 alpha上进行自定义训练，同时我试图向TensorBoard添加一些指标和训练图。考虑下面的人为例子

import tensorflow as tf
from tensorflow.keras.layers import Dense, Input
from tensorflow.keras.models import Model


def create_model():
    inp = Input((32, ))
    net = Dense(16, activation="relu")(inp)
    net = Dense(8, activation="relu")(net)
    net = Dense(2, activation=None)(net)
    return Model(inp, net)


@tf.function
def grad(model, loss, x, y):
    with tf.GradientTape() as tape:
        y_ = model(x)
        loss_value = loss(y_true=y, y_pred=y_)
    return loss_value, tape.gradient(loss_value, model.trainable_variables)


@tf.function
def train_step(model, loss, optimizer, features, labels):
    loss_value, grads = grad(model, loss, features, labels)
    optimizer.apply_gradients(zip(grads, model.trainable_variables))
    return loss_value


def train():
    tf.summary.trace_on(graph=True, profiler=True) …

mystuff.py包含一个模块.我正在使用python版本3.6.

mystuff = {'donut': "SHE LOVES DONUTS!"}

mystuffTest.py包括这个

import mystuff

print (mystuff['donut'])

运行myst​​uffTest.py时收到的错误如下:

$ python3.6 mystuffTrythis.py 
Traceback (most recent call last):
  File "mystuffTrythis.py", line 3, in <module>
    print (mystuff['donut'])
TypeError: 'module' object is not subscriptable

到目前为止,我还没有在stackoverflow上看到这个确切的错误.任何人都可以解释为什么我收到此错误？

我目前正在运行spark 2.1.0.我大部分时间都在PYSPARK shell中工作,但是我需要spark-submit一个python文件(类似于java中的spark-submit jar).你是如何在python中做到的？