Data Machine #201 – Data Machine

by

Small or big? The difference between LMs. AI Giants continue to release bigger, more powerful, often closed LMs. In parallel, I feel that the AI/ML community is moving forward by releasing simpler, smaller LMs that are super efficient enough to provide good results at a much cheaper cost than LLMs. Check out the new Open LLM leaderboard. Is it time to fill up the L? Large language models? Let’s see…

The new Google PaLM 2 model. They claim it achieves SoTA on logic, coding and multilingual tasks. PaLM will power 25 Google products. read more What PaLM 2 can do and how PaLM 2 was built and evaluated

In our research, we’ve learned that it’s not as simple as “bigger is better” and that exploratory creativity is the key to creating great models. Zubin Ghahramani, Vice President of Google DeepMind

It is interesting, The PaLM 2 will be available in 4 sizes from a tiny gecko to a huge unicorn version. Looks like Enrico aka conceptofmind has been busy… He just released An open source implementation of Google PaLM models.

Larger context window icon size. Anthropic announced a massive increase in Claude’s context window to the 100K mark. See. Introducing 100K contextual Windows. Some people say that this type of context window size will reduce the need for vector search DBs and improve summarization, knowledge extraction, CoT and knowledge synthesis. But…

Claude’s 100k tick window test. Jerry @LLamaIndex reviews Claude’s 100K-token model and summarizes what Claude does and doesn’t do well. Jerry says one of the problems is obviously the cost. Claude is expensive. Another problem appears to be poor “reasoning” when using create and refine the prompt in the LLaMA index.

Agents that handle much larger tip sizes, more complex tips. Until recently, the LangChain team followed suit React Pattern:. But now, to address the issue of higher speed sizes and more complex hints, LangChain has just been released Schedule and implement agents. It seems The LangChain architecture is getting pretty big.

New Transformer agents To integrate with 100K+ models. This is pretty huge. Basically, Transformer Agents are a natural language API on top of Transformerswhich allows you to use 100,000+ Hug Face models in all modes: text, image, video, audio, documents… Posted by Thomas a great summary of all Transformer Agents features.

Optimizing LLM Model Learning and Inference. Optimization techniques have been at the heart of ML forever. But I guess LLMs have sharpened the need for new or more advanced techniques for model training and inference optimization. Two great reads below.

Make your ML models smaller, faster, and less data intensive. This post is a nice summary of techniques for reducing model size, training with less memory, reusing existing models, and making efficient use of small datasets.

Arithmetic of transformer inference A few principles that underpin LLM conclusion performance without experiments or hard math. A very simple model of inference delay turns out to fit the empirical results well.

On click. It The deviation principle of information theory was used to address the trade-off between compression and relevant information. Self-supervised learning, in a way, came to the rescue by learning from data without clear labels. However, the optimal purpose of information in SSL remains unclear. Jan Le Koon and Ravid Schwartz have written a new article on all of this.

To click or not to click?. A comprehensive review of the intersection between information theory, self-supervised learning, and deep neural networks.

More on small, efficient models which use techniques such as quantization and adapters. Let me tell you about these two little gems.

Lit-ParrotA new implementation of the SoTA open source language models. Based on the Lit-LLaMA and nanoGPT models, you can run Lit-Parrot on consumer devices. Supports flash focus, LLaMA-adapter, pre-training. Apache 2.0 License!

MLC LLM (updated repo, demo, blog)– MLC LLM is a universal solution that allows anyone to develop, optimize and deploy AI language models natively on any backend, including consumer devices.

Good week!

  1. A study of autonomous agents. semi-technical diving

  2. [free] LLM Bootcamp Spring 2023 (8 videos)

  3. Creating a coding assistant using the StarCoder 16B model

  4. What is Constitutional AI? Introduction:

  5. Guanaco LLM Competition: $1 million prize, June 2023

  6. What’s new in AI/ML @Google? (30 videos, Google I/O)

  7. Abuse of Texts, Maps and Chess Vector Search

  8. Chameleon. Plug & Play Compositional Reasoning with LLMs

  9. AttentionViz. a global view of transformer focus

  10. [lecture] Lekun: On machines that can understand, reason and plan

Share Data Machina with your friends!

  1. School of Ascension. LangChain 101 – Models in Python

  2. LLMTune. 65B 4-bit specification of LLaMAs on a consumer GPU

  3. privateGPT – Chat with your Docs 100% Private, No Internet, No Data Leakage

  1. {flashlight} Shed light on Black Box ML models

  2. Causal Inference with Tree-Based ML Algos

  3. vswift – ML model estimation tools

  1. Kaggle training. Segment Anything Model:

  2. Graphic transformers from scratch (slides, repo)

  3. What is Hyperbolic Deep RL?

  1. MetaAI ImageBind. Complete AI training in 6 ways

  2. VideoChat. E2E Chat-centric video understanding system

  3. Evaluation of APIs implemented for information retrieval

  1. Build contextual DataViz with VizGPT

  2. CosmoGraph. visualize large networks in seconds

  3. Automate Python DataViz annotations with ChatGPT

  1. The ML life cycle for LLMs in practice

  2. SageMaker MLOps pipeline with batch inference

  3. How to build MLOps pipelines in Google Vertex AI

  1. Rewind – AI to give people a perfect memory

  2. Xapien- AI for Automated Background Risk Investigation

  3. Builder – AI to easily build apps without technical knowledge

  1. DiffusionDB – 14 million image hint pairs

  2. TokenMonster. Which symbols best represent the data set?

  3. fastdup – Fast, scalable, image and video dataset analysis

Did you like this post? Tell your friends about Data Machina. Thanks for reading.

Share

Advices. Suggestions: Feedback: letter to Carlos

Compiled by: @ds_ldn: at midnight.



Source link