Guillaume Wenzek 8acb4fe8fb allow enc_kv_cache to be filled up later		1 tahun lalu
..
ci	d80093f9f8 Import ggml to SC	1 tahun lalu
cmake	d80093f9f8 Import ggml to SC	1 tahun lalu
examples	8acb4fe8fb allow enc_kv_cache to be filled up later	1 tahun lalu
include	78718f71e8 [unity.cpp] Clean up conv ops (#259)	1 tahun lalu
scripts	d80093f9f8 Import ggml to SC	1 tahun lalu
src	78718f71e8 [unity.cpp] Clean up conv ops (#259)	1 tahun lalu
tests	d80093f9f8 Import ggml to SC	1 tahun lalu
CMakeLists.txt	31f2419086 Fix unity.cpp ctx management (#177)	1 tahun lalu
LICENSE	d80093f9f8 Import ggml to SC	1 tahun lalu
Makefile	42365dfb74 Cherrypick allocr related changes from public (#247)	1 tahun lalu
README.md	56e5eb146d ggml readme (#170)	1 tahun lalu
build.zig	d80093f9f8 Import ggml to SC	1 tahun lalu
ctypes_utils.py	a768cdf55f Unity inc (#159)	1 tahun lalu
ggml.pc.in	d80093f9f8 Import ggml to SC	1 tahun lalu
ggml.py	b20ffea609 Unity.cpp dec sync (#257)	1 tahun lalu
ggml_convert.py	747002a537 merge with wa_models	1 tahun lalu
requirements.txt	2110c89993 [unity.cpp] make dot/ and test_data folder before the test (#251)	1 tahun lalu
test_ggml_integration.py	f2ef995b95 format/isort	1 tahun lalu
test_unity_cpp.py	b20ffea609 Unity.cpp dec sync (#257)	1 tahun lalu
third_party_ggml.py	b20ffea609 Unity.cpp dec sync (#257)	1 tahun lalu

unity.cpp

Introduction

GGML is an open source library in C to enable large model inference on various hardware platforms. We implemented unity.cpp in ggml. Now it supports SeamlessM4T model for X2T tasks - Speech-to-text translation (S2TT), Acoustic speech recognition (ASR), Text-to-text translation (T2TT).

The project is still active in development. Contributions are welcome!

Build

To build the interactive console for S2TT & ASR,


cd seamless_communication/ggml
mkdir build; cd build
cmake -DGGML_OPENBLAS=ON \
    -DBUILD_SHARED_LIBS=On \
	  -DCMAKE_BUILD_TYPE=Release \
	  -DCMAKE_CXX_FLAGS="-g2 -fno-omit-frame-pointer" \
    ..
make -j4 unity # Interactive Console

For more build commands see Makefile.

CLI usage

Command to launch an interactive console for S2TT & ASR, note that the model already includes vocabulary needed to detokenize.

OPENBLAS_NUM_THREADS=8 ./bin/unity --model seamlessM4T_medium.ggml

In the console, enter the path of local waveform file and target language, separated by space. Note that the first run would include some “warm up” time so could be slow.

Converted ggml models could be downloaded from |SeamlessM4T_large | SeamlessM4T_medium | |-------- | -------- | | model | model |

Fairseq2 model conversion

Models from fairseq2 checkpoints could be converted to ggml automatically with ggml_convert.py.

python ggml_convert.py -m MODEL_NAME

where MODEL_NAME corresponds to asset cards in fairseq2 / seamless_communication, e.g. seamlessM4T_medium, seamlessM4T_large

Python bindings

We also utilize ggml python bindings for better dev experience. For examples of running unity.cpp in python, refer to tests in test_unity_cpp.py.

[Optional]Dependencies

OpenBLAS

We strongly suggest building with OpenBLAS, as we've seen 8x speedup on test machine.

libsndfile

This is needed only for the console to load waveform, but not the library.

README.md