Is that possible to generate texts from OpenAI GPT-2 using TensorFlowJS?
If not what is the limitation, like model format or ...?
-
Have you managed to load it on TFJS?Mohamed Taher Alrefaie– Mohamed Taher Alrefaie2020-07-08 15:30:14 +00:00Commented Jul 8, 2020 at 15:30
-
1I'm having a hard time finding input and output nodes for the solution provided by @frederik-bode. Instead, I'm using "Pytorch serve" to expose the model through Rest API. Also GPT2 model is too large to serve in JS and I'm not seeing any advantage converting to TFJS for my use case.jay– jay2020-07-09 12:30:14 +00:00Commented Jul 9, 2020 at 12:30
-
aaah interesting! Have you done any time profiling to this solution?Mohamed Taher Alrefaie– Mohamed Taher Alrefaie2020-07-09 14:57:37 +00:00Commented Jul 9, 2020 at 14:57
-
1This seems useful: github.com/tensorflow/tfjs/issues/3582Heath Mitchell– Heath Mitchell2021-02-04 14:16:30 +00:00Commented Feb 4, 2021 at 14:16
-
Wait, that uses the answer from hereHeath Mitchell– Heath Mitchell2021-02-04 14:17:45 +00:00Commented Feb 4, 2021 at 14:17
Add a comment
|
2 Answers
I don't see any reason as to why not, other than maybe some operation that is in gpt-2 that is not supported by tensorflowjs.
I don't know how to do it, but here's a nice starting point:
install.sh
python3 -m pip install -q git+https://github.com/huggingface/transformers.git
python3 -m pip install tensorflow
save.py
from transformers import TFGPT2LMHeadModel, GPT2Tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
# add the EOS token as PAD token to avoid warnings
model = TFGPT2LMHeadModel.from_pretrained("gpt2", pad_token_id=tokenizer.eos_token_id)
model.save("./test_gpt2")
that will give you a SavedModel file. Now you can try figure out the input and output nodes, and use tensorflowjs_converter to try and convert it. Pointer: https://www.tensorflow.org/js/tutorials/conversion/import_saved_model.
4 Comments
denislexic
This is in Python, I believe the OP is looking for JS answer.
Heath Mitchell
Yes, this is how to save the existing model from Python so it can be imported into JS
Jules Gagnon-Marchand
any reason why
tensorflowjs is imported?Jules Gagnon-Marchand
removed it because I didn't see one
It's possible. Maybe someone finds this useful in 2023:
One way to achieve this is to convert a TF model with
tensorflowjs-converteras Frederik described (possible problem with this approach is missing custom layers)Use gpt-tfjs - implementation of GPT model in TensorFlow.js. It's possible to load weights directly from HF (example). I developed it to experiment with model training in the browser.
If you just want to generate text without training, you have more options:
- Use transformers.js or ONNX in general. The lib is great and follows Python's
transformerslibrary API. Unfortunately - inference only. - Use ggml + WASM. It's a C/C++ model implementation compiled to WebAssembly (example, talk)
