[AINews] The Dissection of Smaug (72B) • ButtondownTwitterTwitter

The discussions in various AI-related Discord channels continue with a focus on topics like the performance of large models, safety-focused AI models, UI development projects, model merges, fine-tuning methodologies, hardware setups, API changes, compression technologies, and GPT versions. Each community delves into unique challenges and advancements within the AI field, showcasing a diverse range of interests and expertise.

One Size Fits All with Mistral's Subscription:

Users discussed the subscription model for the Mistral Discord chatbot, confirming it is a unified model with payment per token and scalable deployment, highlighted by @mrdragonfox; quantized models, such as those found on Hugging Face, were also mentioned as requiring less RAM.

GPU Economics: Rent vs. Own:

@i_am_dom analyzed the cost-effectiveness of Google GPU rentals versus owning hardware like A100s 40GB, suggesting that after 70000 computational units or about half a year of use, owning GPUs could be more economical.

Docker Deployment Discussion: A request for

TurboPuffer Soars on S3

A new serverless vector database called TurboPuffer was discussed for its efficiency, highlighting warm queries for 1 million vectors taking around 10 seconds to cache. The conversation compared TurboPuffer with LanceDb, noting TurboPuffer's use of S3 and LanceDb's open-source nature.

Podcast Ponders AI and Collective Intelligence

An interview with Yohei Nakajima on the Cognitive Revolution podcast discussed collective intelligence and the role of AI in fostering understanding across cultures.

AI as Google's Achilles' Heel

A 2018 internal Google memo, shared via TechEmails, indicated that the company viewed AI as a significant business risk. This sparked discussions about the ongoing concerns regarding AI's impact on businesses years later.

ChatGPT's Impact on College Processes

The trend of using ChatGPT for college applications was analyzed, referencing a Forbes article that highlighted potential red flags such as the use of specific banned words that may alert admissions committees.

Avoiding Academic Alert with Banned Words

There was a suggestion to program ChatGPT with a list of banned words to prevent its misuse in academic scenarios, particularly in college admissions processes where the use of certain words could trigger alerts.

In this section, discussions revolve around fine-tuning chatbot models and recommendations for dataset selection: - Fine-tuning a base model involves using a good instruct dataset for two epochs, with considerations for dataset sources such as the bagel datasets. - Dataset recommendations for creating an instruct model include WizardLM_evol_instruct_V2_196k, OpenHermes-2.5, and mixing in specific specializations. - The impact of fine-tuning is explored, with insights that it can change a chatbot model's tone and add knowledge, especially when pretraining continues across all layers. - The future of fine-tuning speed and efficiency is discussed, with a resource called unsloth claiming faster and more efficient QLoRA fine-tuning for models like Mistral. - A comparison between fine-tuning and training is made, emphasizing resource savings and stability, while also highlighting the importance of staying updated with fine-tuning trends and specific methods like RHLF with PPO and SFT with DPO. Links to associated GitHub repositories and resources are provided throughout the discussion.

Deep Dive into Deep Learning Breakthroughs:

• User @branchverse shared an article highlighting the progress in deep learning since the 2010s. The article notes innovations driven by open source tools, hardware advancements, and availability of labeled data.

Normcore LLM Reads on GitHub:

• User @husainhz7 linked a GitHub Gist titled 'Normcore LLM Reads,' which is a collection of code, notes, and snippets related to LLMs.

AI-Infused Genetic Algorithm for Greener Gardens:

• User @paccer discussed an article that features a genetic algorithm combined with LLM for gardening optimization. The AI-powered tool GRDN.AI seeks to improve companion planting and is documented in a Medium post.

Unveiling Computer Vision Techniques:

• User @purple_lizard posted a link to the Grad-CAM research paper, which introduces a technique for making convolutional neural network (CNN) decisions transparent via visual explanations.

Exploring AI Research:

• User @kamama2127 pointed out a recent AI research paper on arXiv with a list of authors contributing to the field. The paper discusses new findings and advancements in artificial intelligence.

In the HuggingFace NLP channel, users discussed various topics including troubleshooting saving models with PEFT, seeking JSON-aware LLM for local use, searching for profanity-capable LLM, requesting small models for local code generation, and more. The discussions involved issues with saving models, seeking specific capabilities in Large Language Models, and exploring different model options for specific tasks.

LAION ▷ #general (166 messages🔥🔥):

• User @donjuan5050 voiced concerns about the use of Cascaded ASR + LLM + TTS for speaking bots, favoring end-to-end training using a conversation dataset.

• Legal issues surrounding AI art were discussed, with a court ruling against Midjourney and StabilityAI's dismissal under a First Amendment defense.

• Chatter in the community about sd-forge project avoiding associations with diffusers, automatic1111, and comfyui.

• @thejonasbrothers used recent checkpoints to create detailed Dungeons and Dragons maps, shared by @pseudoterminalx.

• Issues with Hugging Face's services being down highlighted reliance on external APIs.

Links mentioned:

• Speechbot about Bud-e voice assistant
• EvalPlus Leaderboard
• Thumbs Up Double Thumbs Up GIF
• Section 230 - Wikipedia
• Atlas Struts YouTube video
• TheBloke/deepseek-coder-33B-instruct-GGUF · Hugging Face
• GitHub - Ninyago53/webaudiobook
• Forge Is Not Using ComfyUI as A Backend · lllyasviel/stable-diffusion-webui-forge · Discussion #169

The Perplexity AI channel features discussions on various topics such as comparing chatbot models, sensitivity issues with the Perplexity iPad app, and discussions about the Perplexity API. Users also shared their experiences with Perplexity search results, rate limit errors encountered, and inquiries about Mistral 32k context length. The channel also covers issues related to CUDA environments, distributed matrix multiplication, multi-GPU testing, and errors encountered with FAISS embedding vectors.

Addressing @akshay_1’s FAISS error, @uwu1468548483828484 suggested the error might be due to a distributed worker not reaching an allreduce call, causing a timeout. To debug, they recommended running with GDB to inspect which worker hangs.

Spotlight on Automatic Object Detection:

• @pradeep1148 shared a YouTube video titled 'Automatic Object Detection' showcasing zero-shot object detection with the MoonDream Vision Language Model.

Tutorial on Chatting with Documents Using Various Tools:

• @datasciencebasics posted a tutorial video on creating a Retrieval Augmented Generation UI using ChainLit, LangChain, Ollama, & Mistral.

Links mentioned:

• Automatic Object Detection: Video on automatic object detection using zero-shot object detection and MoonDream Vision Language Model.
• Chat With Documents Using ChainLit, LangChain, Ollama & Mistral 🧠: Tutorial video demonstrating the creation of a Retrieval Augmented Generation UI.

To stay updated with the latest AI News, don't forget to subscribe by entering your email and clicking on the 'Subscribe' button in the form provided. You can also connect with AI News on Twitter and check out the newsletter on latent.space. Additionally, you can find AI News on Twitter at latentspacepod and access the newsletter on latent.space. AI News is brought to you by Buttondown, offering the simplest way to initiate and expand your newsletter.