Classification and regression using the same Neural Network using Keras

Question

I would like to build a Neural Network that at the same time output a label for classification and a value for regression. I would like to do that using Keras. Right now my code is only for classification:

 mdl = Sequential()
 mdl.add(Dense(100, activation='relu', input_dim=X_train.shape[1]))
 mdl.add(Dense(200, activation='relu'))
 mdl.add(Dense(100, activation='relu'))
 mdl.add(Dense(6, activation='softmax'))

 mdl.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])

 # early stopping implementation
 filepath="weights.best.hdf5"
 checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, 
 save_best_only=True, mode='max')
 early_stop = EarlyStopping(monitor='val_acc', patience=100, mode='max') 
 callbacks_list = [checkpoint, early_stop]


 # fit network
 history = mdl.fit(X_train, y_train, epochs=2000, batch_size=32, 
 validation_split=0.2, verbose=2, shuffle=True, callbacks=callbacks_list)

So right now I have a softmax activation function on the output layer that correspond to the probability that I use for classification. How can I modify this code to output also a continuos value that will represent my regression problem. I know that Keras Functional API allow to specify multi input and multi output network. Anyone that have an idea on how can I do that?

Answer 1

The same code in a slightly different pattern

There's a straightforward transformation of your code to the Keras Functional API as illustrated in their documentation. You'd need to change your Sequential declaration

mdl = Sequential()
mdl.add(Dense(100, activation='relu', input_dim=X_train.shape[1]))
mdl.add(Dense(200, activation='relu'))
mdl.add(Dense(100, activation='relu'))
mdl.add(Dense(6, activation='softmax'))

to its Functional equivalent:

inputs = Input(shape=(X_train.shape[1],))
layer1 = Dense(100, activation='relu')(inputs)
layer2 = Dense(200, activation='relu')(layer1)
layer3 = Dense(100, activation='relu')(layer2)
classifier = Dense(6, activation='softmax')(layer3)
mdl = Model(inputs=inputs, outputs=classifier)

(often people just re-use the same variable for all the intermediate layers, it's even done in the documentation samples but this IMHO is a bit clearer).

Once you've done that, you can add another output layer that "branches" from the last Dense layer layer3, and set that your model has two outputs, for example:

regression = Dense(1, activation='linear')(layer3)
mdl = Model(inputs=inputs, outputs=[classifier, regression])