为什么TensorFlow matmul()比NumPy multiply()慢得多？

Question

在下面的python代码中,为什么通过numpy进行乘法的时间远小于via tensorflow？

import tensorflow as tf
import numpy as np
import time
size=10000
x = tf.placeholder(tf.float32, shape=(size, size))
y = tf.matmul(x, x)

with tf.Session() as sess:
  rand_array = np.random.rand(size, size)

  start_time = time.time()
  np.multiply(rand_array,rand_array)
  print("--- %s seconds numpy multiply ---" % (time.time() - start_time))

  start_time = time.time()
  sess.run(y, feed_dict={x: rand_array})
  print("--- %s seconds tensorflow---" % (time.time() - start_time))

输出是

--- 0.22089099884 seconds numpy multiply ---
--- 34.3198359013 seconds tensorflow---

Answer 1

好吧,引用文档:

numpy.multiply(x1,x2 [,out])=以元素为单位的乘法参数.

和

tf.matmul(a,b,transpose_a = False,transpose_b = False,a_is_sparse = False,b_is_sparse = False,name = None)

将矩阵a乘以矩阵b,产生a*b.

输入必须是二维矩阵,具有匹配的内部维度,可能在换位之后.

这表明您比较不同的操作:O(n ^ 2)逐点乘法与O(n ^ 3)矩阵乘法.我纠正了测试,在两种情况下使用矩阵乘法2次:

import tensorflow as tf
import numpy as np
import time
size=2000
x = tf.placeholder(tf.float32, shape=(size, size))
y = tf.matmul(x, x)
z = tf.matmul(y, x)

with tf.Session() as sess:
  rand_array = np.random.rand(size, size)

  start_time = time.time()
  for _ in xrange(10):
      np.dot(np.dot(rand_array,rand_array), rand_array)
  print("--- %s seconds numpy multiply ---" % (time.time() - start_time))

  start_time = time.time()
  for _ in xrange(10):
      sess.run(z, feed_dict={x: rand_array})
  print("--- %s seconds tensorflow---" % (time.time() - start_time))

并得到了结果:

--- 2.92911195755 seconds numpy multiply ---
--- 0.32932305336 seconds tensorflow---

使用快速GPU(gtx 1070).