MInimum工作示例tensorflow服务客户端

Question

我正在研究基本的Tensorflow服务示例.我遵循MNIST示例,除了分类我想使用numpy数组来预测另一个numpy数组.

为此,我首先训练了我的神经网络

x = tf.placeholder("float", [None, n_input],name ="input_values")

weights = {
    'encoder_h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])),
    'encoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),
    'encoder_h3': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_3])),
    'decoder_h1': tf.Variable(tf.random_normal([n_hidden_3, n_hidden_2])),
    'decoder_h2': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_1])),
    'decoder_h3': tf.Variable(tf.random_normal([n_hidden_1, n_input])),
}
biases = {
    'encoder_b1': tf.Variable(tf.random_normal([n_hidden_1])),
    'encoder_b2': tf.Variable(tf.random_normal([n_hidden_2])),
    'encoder_b3': tf.Variable(tf.random_normal([n_hidden_3])),
    'decoder_b1': tf.Variable(tf.random_normal([n_hidden_2])),
    'decoder_b2': tf.Variable(tf.random_normal([n_hidden_1])),
    'decoder_b3': tf.Variable(tf.random_normal([n_input])),
}

# Building the encoder
def encoder(x):
    # Encoder Hidden layer with sigmoid activation #1
    layer_1 = tf.nn.tanh(tf.matmul(x, weights['encoder_h1'])+biases['encoder_b1'])
    print(layer_1.shape)
    # Decoder Hidden layer with sigmoid activation #2
    layer_2 = tf.nn.tanh(tf.matmul(layer_1, weights['encoder_h2'])+biases['encoder_b2'])
    print(layer_2.shape)
    # Layer 3
    layer_3 = tf.nn.tanh(tf.matmul(layer_2, weights['encoder_h3'])+biases['encoder_b3'])
    print(layer_3.shape)
    return layer_3


# Building the decoder
def decoder(x):
    # Encoder Hidden layer with sigmoid activation #1
    layer_1 = tf.nn.tanh(tf.matmul(x, weights['decoder_h1'])+biases['decoder_b1'])
    print(layer_1.shape)
    # Decoder Hidden layer with sigmoid activation #2
    layer_2 = tf.nn.tanh(tf.matmul(layer_1, weights['decoder_h2'])+biases['decoder_b2'])
    # Layer 3
    layer_3 = tf.nn.tanh(tf.matmul(layer_2, weights['decoder_h3'])+biases['decoder_b3'])
    return layer_3

# Construct model
encoder_op = encoder(x)
decoder_op = decoder(encoder_op)

# Prediction
y = decoder_op



# Objective functions
y_ = tf.placeholder("float", [None,n_input],name="predict")

接下来有人在这里建议我保存我的网络就像这样..

import os
import sys

from tensorflow.python.saved_model import builder as saved_model_builder
from tensorflow.python.saved_model import utils
from tensorflow.python.saved_model import tag_constants, signature_constants
from tensorflow.python.saved_model.signature_def_utils_impl import     build_signature_def, predict_signature_def
from tensorflow.contrib.session_bundle import exporter

with tf.Session() as sess:
# Initialize variables
    sess.run(init)

    # Restore model weights from previously saved model
    saver.restore(sess, model_path)
    print("Model restored from file: %s" % save_path)

    export_path = '/tmp/AE_model/6'
    print('Exporting trained model to', export_path)
    builder = tf.saved_model.builder.SavedModelBuilder(export_path)


    signature = predict_signature_def(inputs={'inputs': x},
                                  outputs={'outputs': y})

    builder.add_meta_graph_and_variables(sess=sess,
                                         tags=[tag_constants.SERVING],
                                         signature_def_map={'predict': signature})

    builder.save()


    print 'Done exporting!'

接下来,我按照说明在localhost:9000上运行我的服务器

bazel build //tensorflow_serving/model_servers:tensorflow_model_server

我设置了服务器

bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_base_path=/tmp/AE_model/

问题

现在我想编写一个程序,这样我就可以将eclipse中的C++程序中的Mat向量(我使用很多库)传递给我的服务器,这样我就可以进行某种预测.

我首先考虑使用inception_client.cc作为参考.但是,似乎我需要Bazel来编译它,因为我无法在任何地方找到prediction_service.grpc.pb.h :(

所以似乎我唯一的另一个选择是使用python调用脚本我得到以下输出:

<grpc.beta._client_adaptations._Rendezvous object at 0x7f9bcf8cb850>

任何有关此问题的帮助将不胜感激.

谢谢.

编辑:

我重新安装了protobuf和grpc并按照建议运行命令:

我的命令有点不同,我不得不在我的服务文件夹之外使用它(在Ubuntu 14.04中).

sudo protoc -I=serving -I serving/tensorflow --grpc_out=. --plugin=protoc-gen-grpc=`which grpc_cpp_plugin` serving/tensorflow_serving/apis/*.proto

这会生成.gprc.pb.h文件,然后将它们拖到/ apis /文件夹中,错误就会消失.现在我收到了错误

/tensorflow/third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1:42: fatal error: unsupported/Eigen/CXX11/Tensor: No such file or directory

即使这个文件确实存在.任何建议表示赞赏.

谢谢@subzero!

编辑2

通过更新到最新的Eigen版本并从源代码构建,我能够解决Eigen的问题.接下来我指向/ usr/local/include/eigen3 /

之后我遇到了tensorflow库的问题.这些我通过使用lababidi的建议生成libtensorflow_cc.so库来解决. https://github.com/tensorflow/tensorflow/issues/2412

我有最后一期.一切似乎都很好,除了我得到错误:

未定义的引用`tensorflow :: serving :: PredictRequest :: ~PredictRequest()'

似乎我缺少链接器或库.有谁知道我错过了什么???

Answer 1

自定义客户端和服务器的示例：

添加到张量流模型的服务器代码：

import grpc
from concurrent import futures
import python_pb2
import python_pb2_grpc

class PythonServicer(python_pb2_grpc.PythonServicer):


    def makePredictions(self, request, context):


        items = eval(str(request.items)) #Receives the input values for the model as a string and evaluates them into an array to be passed to tensorflow

        x_feed = items

        targetEval_out = sess.run(confidences, feed_dict={x:x_feed}) #"confidences" is the output of my model, replace it for the appropriate function from your model


        out = str(targetEval_out.tolist()) #The model output is then put into string format to be passed back to the client. It has to be reformatted on the other end, but this method was easier to implement

        return python_pb2.value(name=out)


print("server online")
MAX_MESSAGE_LENGTH = 4 * 1024 * 1024 #can be edited to allow for larger amount of data to be transmitted per message. This can be helpful for making large numbers of predictions at once.
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10), 
options=[('grpc.max_send_message_length', MAX_MESSAGE_LENGTH), (
'grpc.max_receive_message_length', MAX_MESSAGE_LENGTH)])
python_pb2_grpc.add_PythonServicer_to_server(
PythonServicer(), server)
server.add_insecure_port('[::]:50051')
server.start()

客户端C++代码：

#include <grpc/grpc.h>
#include <grpc++/channel.h>
#include <grpc++/client_context.h>
#include <grpc++/create_channel.h>
#include <grpc++/security/credentials.h>
#include "python.grpc.pb.h"

using grpc::Channel;
using grpc::ClientContext;
using grpc::ClientReader;
using grpc::ClientReaderWriter;
using grpc::ClientWriter;
using grpc::Status;
using python::request;
using python::value;
using python::Python;

using namespace std;


unsigned MAX_MESSAGE_LENGTH = 4 * 1024 * 1024; //can be edited to allow for larger amount of data to be transmitted per message. This can be helpful for making large numbers of predictions at once.
grpc::ChannelArguments channel_args;
channel_args.SetMaxReceiveMessageSize(MAX_MESSAGE_LENGTH);
channel_args.SetMaxSendMessageSize(MAX_MESSAGE_LENGTH);

shared_ptr<Channel> channel = CreateCustomChannel("localhost:50051", grpc::InsecureChannelCredentials(),channel_args);
unique_ptr<python::Python::Stub>stub = python::Python::NewStub(channel);

request r;
r.set_items(dataInputString); //The input data should be a string that can be parsed to a python array, for example "[[1.0,2.0,3.0],[4.0,5.0,6.0]]"
//The server code was made to be able to make multiple predictions at once, hence the multiple data arrays 
value val;
ClientContext context;

Status status = stub->makePredictions(&context, r, &val);

cout << val.name() << "\n"; //This prints the returned model prediction

python.proto代码：

syntax = "proto3";


package python;

service Python {

    rpc makePredictions(request) returns (value) {}


}

message request {
  string items = 1;
}


message value {
  string name = 1;
}

我不确定这些代码片段是否可以单独工作，因为我刚刚从当前项目中复制了相关代码。但希望这能够为需要张量流客户端和服务器的任何人提供一个良好的起点。