• The Deep View
  • Posts
  • ⚙️ Meta releases its biggest generative AI model ever

⚙️ Meta releases its biggest generative AI model ever

Good morning. The first two entries in this quarter’s Big Tech earnings season went up to bat last night, with Google meeting analyst expectations and Tesla falling short.

In other news, Meta released its largest generative AI model ever in the form of Llama 3.1 405B. We break it all down for you below.

— Ian Krietzberg, Editor-in-Chief, The Deep View

In today’s newsletter:

AI for Good: Enhanced carbon sequestration

Source: Unsplash

The goals of a global response to climate change are really centered around two main things: first, stop emitting carbon into the atmosphere, and second, start getting rid of the carbon that’s already there. 

The other method involves geological sequestration, in which CO2 is captured, pressurized into a liquid and subsequently injected deep underground. 

This process is traditionally time-consuming, as scientists need to carefully model where they are injecting this carbon to avoid untenable pressure buildups. 

The details: Researchers at the Heriot Watt University recently unveiled an AI-based simulator designed to make this process of carbon sequestration and storage more efficient and economically viable. 

  • The University’s Eco-AI project claims this new simulator can slash the time needed for modeling carbon storage from 100 days to 24 hours. 

Why it matters: “Our research has the ability to really advance existing scientific research streams to source suitable options for safe storage of CO2 without consuming too much energy and without the need to deploy expensive and often time-consuming exploratory investigations,” Professor Ahmed H. Elsheikh, a leader of the effort, said in a statement. 

Transform Your Business with Kittl AI: The Ultimate Graphic Design Solution

Struggling with graphic design? Lack the skills, budget, or time to hire a professional designer? Kittl AI is here to revolutionize your workflow.

Kittl is an intuitive yet powerful graphic design platform powered by the groundbreaking Kittl AI – the first generative AI tailored for graphic designers. Say goodbye to the limitations of static-generated images. With Kittl, you can remove and change backgrounds, fully customize colors, and integrate your designs seamlessly in just seconds, all within one platform.

What makes Kittl stand out?

  • 10,000+ Professional Templates: Crafted by world-class designers for any use case—marketing assets, pitch decks, charts, infographics, and more.

  • Advanced AI Tools: Streamline your design process by 75% with Prompt-to-Image, Prompt-to-Vector, Background Remover, Vectorizer, and Upscaler.

  • Kittl AI Co-Pilot: Your personal design assistant, enhancing your creations in seconds.

Experience the future of design today. Try Kittl for free and use code “DEEP25” to get a 25% discount on all premium plans.

In Taiwan, farmers had to give up their water to AI companies

Source: Taiwan Semiconductor Manufacturing Co., Ltd.

The issue of AI and sustainability is a two-pronged problem. The first prong is all about excessive energy use. The second is all about excessive water use.

  • Dr. Shaolei Ren recently found that “the global AI demand may be accountable for 4.2 – 6.6 billion cubic meters of water withdrawal in 2027.

  • Ren said recently that “the strain on local freshwater resources imposed by the substantial water consumption associated with AI, both directly for onsite server cooling and indirectly for offsite electricity generation, can worsen prolonged droughts in water-stressed regions.”

This scenario has played out in Taiwan, home of the Taiwan Semiconductor Manufacturing Company (TSMC), a prominent global semiconductor. 

The details: Taiwan has been faced with a severe drought since mid-2020 (that might finally be starting to ease up). 

  • 2023 marked the third year in a row that the country paid its farmers subsidies not to grow rice so that nearby semiconductor factories could tap into scarce water supplies for chip-making purposes. 

  • Yang Kuanwei, a farmer in southern Taiwan, told NPR at the time: “We barely have enough water, and you're diverting even more for others to use.”

In light of this, TSMC has been researching methods of better water management and recycling; in 2022, it opened its first industrial water recycling plant. It said the plant will provide 10,000 metric tons of reclaimed water daily by the end of 2022. 

  • The Backlash Against AI Scraping Is Real and Measurable (404 Media).

  • AI is already taking jobs in the video game industry (Wired).

  • What Google’s decision to keep cookies means for the internet (CNBC).

  • Apple moves forward with a foldable iPhone (The Information).

  • Google meets earnings expectations (CNBC) while Tesla falls short as auto gross margins fall (CNBC).

The FTC has a problem with surveillance pricing

Source: Unsplash

The U.S. Federal Trade Commission (FTC) on Tuesday issued orders for information to eight companies — Mastercard, Revionics, Bloomreach, JPMorgan Chase, Task Software, PROS, Accenture, and McKinsey & Co. — that offer surveillance pricing products and services built on consumer data. 

The details: The FTC’s goal is to better understand the way surveillance pricing is impacting consumers. Its list of companies here have all advertised their use of AI and other technologies to target prices for individual consumers. 

  • The FTC said that “many consumers today are not actively aware that their devices constantly gather data about them, and that that data can be used to charge them more money for products and services.

  • “An age-old practice of targeted pricing is now giving way to a new frontier of surveillance pricing.”

The FTC said it would specifically be looking into the details of data collection and inputs as it wades into an environment where pricing algorithms have become a common practice

Invest smarter with Public and AI

If you're looking to leverage AI in your investment strategy, you need to check out Public.

The all-in-one investing platform allows you to build a portfolio of stocks, options, bonds, crypto and more, all while incorporating the latest AI technology — for high-powered performance analysis — to help you achieve your investment goals.

Join Public, and build your primary portfolio with AI-powered insights and analysis.

Meta releases its biggest genAI model ever

Source: Meta; Created with AI by The Deep View

Meta on Tuesday released its latest generative AI model, a system called Llama 3.1. Meta claims that the model represents the world’s “first frontier-level open source AI model.”

Meta said that it tested the model against the competition in 150 different benchmark tests, finding that Llama 3.1 is “competitive” with GPT-4, GPT-4o and Claude 3.5 Sonnet; in certain benchmarks, Meta said that its newest model outperformed the competition. 

The details: Meta will be enhancing Meta AI with Llama 3.1. CEO Mark Zuckerberg on Tuesday said that Meta AI is on track to “become the most-used AI assistant in the world by the end of the year.” 

  • The largest version of Llama 3.1 has 405 billion parameters and was trained on 16,000 Nvidia H100 GPUs. Meta didn’t disclose the cost of training Llama 3.1, but based on the cost of chips alone, it was likely in the region of the hundreds of millions of dollars.

  • The system is launching with 25 partners, including AWS, NVIDIA, Databricks, Groq, Dell, Azure and Google Cloud, all of which will be offering service on day one of launch. 

In the blog post that heralded the arrival of this new model, Meta doubled down on the idea of open-source, saying that “openness drives innovation” and that open-source models prevent the concentration of power in the hands of the few, rather than the many. 

  • But details on the model’s training data and code remain elusive; as with Llama 2 and 3, Llama 3.1 seems to be more ‘open-weights’ than truly ‘open-source.’ 

Zuckerberg addressed safety concerns in a letter published alongside the release of the model. He said that Meta’s “safety process includes rigorous testing and red-teaming to assess whether our models are capable of meaningful harm, with the goal of mitigating risks before release.”

Meta said that it conducted “extensive” red-teaming of the model with internal and external experts to “stress test the models and find unexpected ways they may be used.”

AI researcher Gary Marcus said the release was “huge” for a few reasons: 

  • Since it is seemingly on par with GPT-4, but is more open than ChatGPT, it represents a significant challenge to OpenAI’s closed business model.

  • And despite the enormous size and cost of the model, it is not significantly better than other GPT-4-level models: “Sure looks (like) deep learning is hitting a wall, pending some kind of new breakthrough,” Marcus said.

Meta’s model evaluation benchmarks for Llama 3.1 405B (Meta).

Concerns about open-source aside, I would like to call your attention to a research paper on ‘open-washing’ published in March of this year. 

  • The paper found that the release-by-blog post method that has become so popular in the AI sector allows companies to “retain the veneer of scientific work while at the same time avoiding the fine-grained accounting and the scrutiny of peer review that comes with actual scientific publication.”

  • And, as that paper goes on to discuss, both Llama 2 and Llama 3 aren’t as open as Meta has claimed — the researchers described the models as, “at best open weights, and closed in almost all other aspects.”

Llama 3.1 seems to be continuing that same trend; though it talked a lot about open-source (the term was used more than 40 times across its blog and Zuckerberg’s letter), the model family doesn’t appear to be more than open-weights.

Meta didn’t respond to a request for comment on this point.

It’s also worth noting that the training process for the Llama 3.1 model family produced 11,390 tons of carbon dioxide, according to Meta, which, according to Dr. Sasha Luccioni, is “orders of magnitude more than previous generations of models (eg 100T for BLOOM, 540T for Llama 2).”

Which image is real?

Login or Subscribe to participate in polls.

A poll before you go

Thanks for reading today’s edition of The Deep View!

We’ll see you in the next one.

Your view on Sam Altman’s UBI:

First, your answers to this question were fascinating to read and I really appreciate the thought you all put into this issue. Wish I could feature everybody!

A quarter of you don’t trust tech companies to make it work; 20% of you think it would be absolutely dystopic and another 20% of you think it’s not needed and wouldn’t work anyway. The remainder are excited about — or resigned to — UBI as our only option.

Dystopia:

  • “I worked on a study on UBI for a UK government think tank 25+ years ago, and we came to the same conclusion. How could it be equitably funded? Before we solve the technical problems of AGI we have to solve the social and political problems.”

It’s the only option:

  • “We have to shape the public discourse in a way that would demand UBI as a human right not as a favor of tech companies ”

How do you feel about open-source AI models?

Login or Subscribe to participate in polls.

*Public disclosure: All investing involves the risk of loss, including loss of principal. Brokerage services for US-listed, registered securities, options and bonds in a self-directed account are offered by Public Investing, Inc., member FINRA & SIPC. Cryptocurrency trading services are offered by Bakkt Crypto Solutions, LLC (NMLS ID 1828849), which is licensed to engage in virtual currency business activity by the NYSDFS. Cryptocurrency is highly speculative, involves a high degree of risk, and has the potential for loss of the entire amount of an investment. Cryptocurrency holdings are not protected by the FDIC or SIPC.

Alpha is an experiment brought to you by Public Holdings, Inc. (“Public”). Alpha is an AI research tool powered by GPT-4, a generative large language model. Alpha is experimental technology and may give inaccurate or inappropriate responses. Output from Alpha should not be construed as investment research or recommendations, and should not serve as the basis for any investment decision. All Alpha output is provided “as is.” Public makes no representations or warranties with respect to the accuracy, completeness, quality, timeliness, or any other characteristic of such output. Your use of Alpha output is at your sole risk. Please independently evaluate and verify the accuracy of any such output for your own use case.