AlphaSignal

Hey ,

Welcome to this week's edition of AlphaSignal the newsletter for AI experts.

Whether you are a researcher, engineer, developer, or data scientist, our summaries ensure you're always up-to-date with the latest breakthroughs in AI.

Let's get into it!

Lior

^HIGHLIGHT
⚡Lit-Parrot

Lit-Parrot, is an open-source language model repository powered by Lightning Fabric⚡ and built upon the Lit-LLaMA and nanoGPT.

It features implementations of state-of-the-art open-source large language models (LLMs) including:

• StabilityAI StableLM

• EleutherAI Pythia

• TII UAE Falcon

• Together RedPajama-INCITE

The repository provides user-friendly scripts to fine-tune pretrained models on the instruction tuning Alpaca dataset, utilizing parameter efficient techniques such as:

• LLaMA Adapter: A prefix-tuning technique augmenting the attention blocks in a language model.

• LLaMA Adapter v2: An enhanced version of the adapter that adds trainable parameters to each transformer layer.

• Low-rank adaptation (LoRA): A method breaking down weight updates into smaller, trainable units, enhancing efficiency and speed.

There is a guide on how to run large models on various (consumer) hardware by using quantization techniques. This, along with the support of flash attention, adds to its utility. The core design principle is clarity and openness, emphasizing readability and hackability. Released under Apache 2.0 license, Lit-Parrot truly exemplifies the principle of open-source knowledge sharing!

Access an index of billions of pages with a single API call.

If your work involves AI, then you know the overwhelming need for new data. Your competitors might be building incredible products…but if they’re all using the same datasets to train their models, then they’re at a disadvantage.

The Brave Search API gives you access to an independent, global search index to train LLMs and power AI applications.

Brave Search is the fastest-growing search engine since Bing, and it’s 100% independent from Big Tech. Its index features billions of pages of high-quality data from real humans - and it’s constantly refreshed thanks to being default in the Brave browser.

Get started testing the API for free:

⚙️ TOP OF GITHUB

TransformerOptimus/SuperAGI
A dev-first open source autonomous AI agent framework, enabling developers to build, manage & run useful autonomous agents quickly and reliably.

ggerganov/ggml
GGML is a C library for machine learning which makes use of a technique called "quantization" that allows for large language models to run on consumer hardware.

DAMO-NLP-SG/Video-LLaMA
Video-LLaMA is an instruction-tuned audio-visual language model for video understanding. This project is working on empowering large language models with video and audio understanding capability.

shi-labs/matting-anything
Matting Anything Model (MAM) is an efficient and versatile framework for estimating the alpha matte of any instance in an image with flexible and interactive visual or linguistic user prompt guidance.

AI4Finance-Foundation/FinGPT
Open-source financial LLMs, developed to democratize financial data, by offering an accessible alternative to proprietary models.

^{PYTORCH TIP}
Auto-Scale Batch Size

Auto-scaling the batch size is a technique to automatically find the largest batch size that fits into memory for a given model and data. This is beneficial because larger batch sizes can result in faster training times due to more efficient use of hardware resources (GPU). However, the downside is that they can also result in memory overflow errors if the batch size is too large for the GPU to handle.

Within PyTorch framework, TOMA is an approach that can be used for auto-scaling batch sizes. Specifically, it retries code that fails due to OOM (out-of-memory) conditions and lowers batch sizes automatically (e.g. from 512, to 256, then 128, etc.). To avoid failing over repeatedly, a simple cache is implemented that memorizes that last successful batchsize given the call and available free memory.

Link: https://github.com/BlackHC/toma 

pip install toma

from toma import toma


@toma.batch(initial_batchsize=512)
def run_inference(batchsize, model, dataset):
	# your inference code


run_inference(batchsize, model, dataset)

Want to promote your company, product, job, or event to 100,000+ AI researchers and engineers? You can reach out here.

Thank You

Want to promote your company, product, job, or event to 100,000+ AI researchers and engineers? You can reach out here.