how can I make addition inner layer in tensorflow

Question

New to tensorflow (and ai in general) - I am trying to build the simplest feed-forward model using tensorflow. I would like to run a sequential model on the fashion mnist data. The only complication I am trying to perform is having some kind of middle layer that adds the former layer with the inputs.

I tried the following:

import tensorflow as tf
import numpy as np

fashion_mnist = tf.keras.datasets.fashion_mnist

(x_train, y_train) , (x_test, y_test) = fashion_mnist.load_data()

x_train, x_test = x_train/255.0, x_test/255.0

input_layer = tf.keras.layers.Flatten(input_shape=(28,28))
inner_layer = tf.keras.layers.Dense(28*28, activation="relu")

hidden_layer_01 = tf.keras.layers.Dense(16, activation="relu")
hidden_layer_02 = tf.keras.layers.Dense(16, activation="relu")
hidden_layer_03 = tf.keras.layers.Dense(16, activation="relu")
hidden_layer_04 = tf.keras.layers.Dense(16, activation="relu")

hidden_layer_add = tf.keras.layers.add(28*28)([input_layer, inner_layer])

hidden_layer_dropout = tf.keras.Dropout(0.1)

output_layer = tf.keras.layers.Dense(10)

model = tf.keras.Model(
    input_layer,
    hidden_layer_01,
    hidden_layer_02,
    inner_layer,
    hidden_layer_add,
    hidden_layer_03,
    hidden_layer_04,
    hidden_layer_dropout,
    output_layer
)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

model.compile(optimizer='adam',
              loss=loss_fn,
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=10)

print("EVALUATING...")
model.evaluate(x_test,  y_test, verbose=2)

I get:

Traceback (most recent call last): File "D:\PROJECTS\Python\pythonProject\fashionMnist01\main.py", line 30, in hidden_layer_add = tf.keras.layers.add(28*28)([input_layer, inner_layer]) File "D:\PROJECTS\Python\pythonProject\fashionMnist01\venv\lib\site-packages\keras\layers\merging\add.py", line 92, in add return Add(**kwargs)(inputs) File "D:\PROJECTS\Python\pythonProject\fashionMnist01\venv\lib\site-packages\keras\utils\traceback_utils.py", line 70, in error_handler raise e.with_traceback(filtered_tb) from None File "D:\PROJECTS\Python\pythonProject\fashionMnist01\venv\lib\site-packages\keras\layers\merging\base_merge.py", line 84, in build if not isinstance(input_shape[0], tuple): IndexError: tuple index out of range

How can I make this work?

Innat · Accepted Answer · 2023-05-15 20:53:34Z

There are some issue in your model definition. It looks like you're mixing functional and sequential APIs in your model definition. Please check the documentation of functional and sequential APIs to build the model. There are mainly three ways (3 APIs) to build the model in keras, functional and sequential APIs are 2 of them. If you're wondering the difference of this APIs, check this blog post.And here is one possible way to build your broken model properly in functional API.

import tensorflow as tf
from tensorflow import keras

input = keras.Input(shape=(28, 28))
x = keras.layers.Flatten()(input)
x = keras.layers.Dense(28*28, activation="relu")(x)

hid_a = keras.layers.Dense(16, activation="relu")(x)
hid_b = keras.layers.Dense(16, activation="relu")(hid_a)
hid_add = keras.layers.Concatenate()([hid_b, x])
hid_c = keras.layers.Dense(16, activation="relu")(hid_add)
hid_d = keras.layers.Dense(16, activation="relu")(hid_c)

hid_drop = keras.layers.Dropout(0.1)(hid_d)
output_layer = keras.layers.Dense(10)(hid_drop)

model = keras.Model(inputs=input, outputs=output_layer)
keras.utils.plot_model(model, show_shapes=True)

Collectives™ on Stack Overflow

how can I make addition inner layer in tensorflow

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related