Is Generative AI Plateauing?, Booming Bluesky, Apple’s Smart Glasses Play

Channel: Alex Kantrowitz

Published at: 2024-11-15

YouTube video id: 2zcN_PpChdk

Source: https://www.youtube.com/watch?v=2zcN_PpChdk

is generative AI plateauing as training
methods top out blue sky is booming as
an alternative social network and apple
looks into smart glasses all that more
is coming up on a big technology podcast
Friday edition right after this welcome
to Big technology podcast Friday edition
where we break down the news in our
traditional cool-headed and nuanced
format we have a great show for you
today covering everything happening in
the world of AI very big news there's
concern that training methods that have
gotten the generative AI field to hear
are not going to continue to scale and
that's really coming to the full right
now we're going to talk about that we're
also going to talk about the rise of
Blue Sky whether it will be sustainable
and apple smart glasses play which is
quite interesting it might be coming
soon joining us as always on Fridays to
break it all down is Ron joh Roy of
margins Ron John welcome to the show
great to see you scaling laws are here
Alex they've finally come for the
industry I know you're excited about
what's going to happen with this but I
know you're even more excited about the
Jake Paul m Tyson fight who you
got neither neither I think Netflix is
genius in promoting their live
programming by just bringing out two
people that no one wants to see win but
I'll still take Tyson if I have to all
right I'm taking Paul we can put it on
the prediction markets and see what
happens my friend group has been saying
that this is an exhibition match
basically and a lot of betting sites are
looking at it the same way the folks are
saying that it's rigged do you think
this is rigged or a real
fight I think this is is a real fight I
don't think Netflix would go into this
fully rigged or making it reality TV but
I think it's a good reminder of the
Blurred Lines between reality TV and uh
actual live programming but I think it's
real I'm going to go rigged okay so we
got a lot to Market contracts to to set
up right now absolutely and now we can
talk a little bit about what's happening
in the AI world where shall we say
there's another fight going on between
the purists that believe that language
models will continue to scale if you add
more data and compute and power to the
mix and those that say eventually these
models are going to hit a wall there's
been a long Brewing battle between these
two factions and this week I think has
been the week where both sides have
started to put their positions in the
ground and say you know what I've won
and that's really come on the back of
this great information report that talks
about how open AI has basically found
that it has hit the limit of
improvement when it trains with more
data more compute and more power here's
from the story The number of people
using chat GPT and artificial
intelligence projects products is
soaring the rate of improvement for
basic building blocks underpinning these
products is slowing down the challenge
that openai is experiencing with its
upcoming Flagship model code named Orion
shows what the company is up against
while Orion's performance and ended up
exceeding that of Prior models the
increase in quality was smaller compared
to the jump between its last models gpt3
and GPT 4 some reearch some researchers
at the company believe Orion isn't
reliably better than its predecessor in
handling certain tasks this could be a
problem as Orion may be more expensive
for open AI to run its data centers in
its data centers compared to other
models basically the idea here is
that sorry one sec basically the idea
here is that open AI has been training
subsequent foundational models with more
data more compute and it's reached the
point of diminishing returns to the
point that this Grand next model that
it's supposed to release we don't even
have a sniff of GPT 5 yet it's calling
this Orion is going to be just a bit
better but more expensive Plateau hit
the folks who say that the scaling isn't
going to work have claimed Victory
meanwhile open a and Sam Alman are
saying not just yet what do you make of
this battle I am happy about this I
think I've talked a lot about how I
don't need gp5 just yet I think the
amount of opportunity there is around
actually productizing the current models
is so massive right now again like
there's so many little magical moments
even with Claude with Chachi PT with any
of these tools that you see so much
potential but then actually Translating
that into helping you do your job better
or create certain things better I think
that there's there there's just so much
work to be done there that everyone
competing to kind of create this next
massive foundational model has never
made a ton of sense to me before they
actually just got a gp4 or uh Claude 3.5
Opus working you know kind of pushing it
to its limits and making it work as well
as possible so I'm kind of hoping that
this actually moves people towards
making tools people use rather than just
saying AGI and GPT 5 and what whatever
else now look I I hear you but I also
have to disagree I mean The Field's
potential is so much more if these
models continue to improve and while
they're good today they're not um where
they've been promised to be and if this
is the limit then it severely diminishes
sort of the potential of these models to
change everything we do as the AI
industry has been promised and it's by
the way it's not just um this
information report lots of people have
been saying this so this is uh elas
talked about it uh he says to reuter
results of from scaling up pre-training
the phase of training an AI model that
uses a vast amount of unlabeled data to
understand language patterns and
structures it's plateaued Ben Horwitz
and Mark andreon talking on their
podcast hor it says we're increasing the
number of Graphics processing units used
to train AI uh but we're not getting the
intelligent improve intelligent
improvements at all out of it and Andre
has been saying that lots of smart
people are working on breaking through
the ASM toote figuring out how to get to
higher levels of reasoning capability I
mean shouldn't we just put your concern
your concern about building practical
applications aside for a moment I don't
think anyone's going to disagree that
it's a time to build practical
applications but isn't this like fairly
concerning for the progress of the AI
industry if let's say this is about as
smart as they're going to get no I think
first of all I love Ilia Unleashed right
now going on going to writers and now
being able to say things like uh the
vast amounts of unlabeled data to
understand language and patterns and
structures have plateaued um but I yeah
I think the focus being kind of
distracting by focusing only on these
step changes in quality of the models
has distracted from practical
applications yes we should be able to
have both but you even see it in the way
that an open AI is structured as a
company and where they invest their
resources we talked a lot about this
with Corey Weinberg from the information
that the cost structure of open aai is
still much more heavily towards the R&D
and improving the actual models versus
building out a good Salesforce and a
sales enablement and customer success
team and these things might sound boring
but if you're actually want these these
Technologies to be adapted by
corporations and companies and just
everyday people they have to be easier
to use and more practical like I I I get
the idea that there's a lot of times
that if you're using one of these tools
it doesn't work perfectly the first time
and everyone the kind of natural
reaction is okay I guess it's not good
enough but then you learn the better you
prompt it the different you structure
your workflow you can get it to do what
you want it to do but instead I think a
lot of these companies are promising the
model will get so smart in the next
iteration that you don't even have to do
that work around prompting and workflow
building that it'll just figure it out
and it'll be okay and AGI will be here
Etc but isn't that what they're trying
to do I mean isn't aren't you ignoring
the business story here that open a just
raised the largest VC round in history
$6 billion uh is it Microsoft or Amazon
are hooking up to nuclear power plants
anthropic is out in the market trying to
raise billions of its own like just from
a business standpoint if these companies
cannot Advance this anymore isn't all
that money going to come do and sort of
crumble the industry I'm not ignoring
the business story at all that is the
business story to me uh I think like
over raising for the R&D side of things
rather than the actual like
operationalization and building out
businesses on top of the existing
technology I mean again we've debated
this plenty I think that is a huge
mistake and that it actually you know
potentially hampers the long-term
development of the industry so if it
actually means this slowdown you know
puts a little cold water on the Promises
of gp5 and whatever else and people just
get back to work in terms of actually
building things that solve problems I'm
happy about that I'm trying to pin you
down here a little bit on the technology
question and you keep wiggling your way
out which I respect but I have to no let
me let me ask it I have to ask like
isn't there just a tad of of
disappointment on your end if this is
sort of the end here the end of the road
in terms of where this is not at all I
mean the things I've already been able
to do I I just made a game in Claude the
other day I saw some video of like uh
it's kind of like
a Space Invaders type of game I coded a
Space Invaders type game with these like
custom images myself in an hour and then
hit the cloud limit which a lot of
listeners probably do and it's kind of
annoying even as a paying customer but
like that was magical to me and that
exists on the existing technology it's
possible and there's so many other
applications I can imagine if I'm able
to do that for fun in an hour that are
not being properly explored because all
the attention and hype is on the much
much bigger thing so if the Claud
business gets built is you know actually
teaching people how to use the existing
technology well I think that's has again
much better longer term potential than
the entire bet on the entire industry is
the technology will get step change
better in the next year or
two yeah I don't know about that I mean
they have that has been the BET though
so we'll see it has it has and I don't
think it's the right one and I think
something that pushes us away from that
kind of strategy is going to be a good
it'll shake things up it'll definitely
shake things up but I think it's it's
healthy for the longer term uh world of
AI you know you've really not played
into my game today where I wanted to
evoke feeling one single feeling of
disappointment or sadness from you and
you say yeah it would be tough if this
is how I'm feeling gosh like if this
promised uh you know AI Revolution ends
here then I don't know how far we get uh
and just go like OG Shucks and then I
come in and say well actually maybe
we're not done you know sort of like
that Walter White GI we're done when we
say I'm done all right well I'll give
you video generation
is the one area that I do think we are
severely we're not even close to
anything
interesting and we've been promised
things that are interesting I.E Sora um
but we're very very far away even the
runway MLS and other tools that I've
tried we're so that's one area where I
see a huge need for technological
Improvement but for any content
generation any coding even data analysis
right now
I think the models are pretty damn good
at doing what most people need them to
do we just don't know most people just
don't know how to use them correctly
Well Ron John thank you for playing
along and I have to inform you we're not
done we're done when we say I'm
done and that is because that is
because yes these research houses might
have hit some sort of wall and the
reason why they're hitting the wall is
obvious that they're using synthetic
data because they've run out of data and
it's offering uh less good results and
this has sort of been the issue with
these training these new models however
in recent times there has been a
development of a new discipline here
which is reasoning and we talked about
it back in the day and that's what sort
of freaked Elia out and he left open Ai
and that really might be the F the near
future of this um of this field where
the models now such as open AI 01 are
prompted and they think and the more
they think the better they get and uh
this is again from the information in
open AI case researchers have developed
a type of reasoning Model A one that
takes more time to think about the data
that llm trained on before spitting out
an answer this means the quality of 0's
responses can continue to improve when
the model is provided with additional
Computing resources while it's answering
user questions even without making
changes to the underlying model and
Casey Newton from platformer cited one
example from one an open a researcher
talking about it this open researcher
says and this was out of Ted AI talk it
turns out that having the bot think for
just 20 seconds in a hand of Poker step
by step got the same boosting
performance as scaling up the model by
100,000 times and training it for
100,000 times longer so I think what
we're about to see is a pivot in the AI
research field where yes they might be
applying practically some of the models
that exist today but it seems to me like
everybody is going to go completely in
on this reasoning format and that is
going to be where we see the
improvements and that's why I want to
highlight this post from uh Dan shipper
I saw that I saw this week he says the
message that the um information
headlines conveys is at odds with what
people inside the big labs are actually
feeling and saying it is technically
correct but the takeaway for the Casual
reader that AI progress is slowing is
the exact opposite of what I'm hearing
so this might be a combination of Spin
and reality but I'm curious how much
stake you're putting into reasoning when
it comes to being able to advance the
status quo yeah no I think both
reasoning and how synthetic data is used
matter and I think actually are an
almost more promising direction for the
industry than just raw processing powder
Power and size I think first on the
synthetic data like we're going to be
talking about a company writer.com in
just a little bit but they one of the
things they did was like create their
own Foundation models and they
apparently trained them for
$700,000 total by using really targeted
synthetic data to create different
models for different kinds of problems
and I think in the coming months and
years we're going to start to see some
awkward headlines around Size Matters
because smaller will be better in terms
of actual models being used and like
again one model should not be reliable
to solve every problem for everyone at
all times versus maybe there is a model
focused on financial data analysis and
it's it's actually much better at
solving problems around that versus
writing poetry or generating images so I
think using really targeted synthetic
data for more targeted models is
actually a really interesting space in
terms of the actual reasoning side I
think that's really interesting like
like rather than coming up with new ways
of actually generating the answers using
the existing information that could be
incredibly promising and solv so many
the ideas so many so many of the
challenges facing the industry like cost
for any kind of New Foundation model
like just you know viability of these
things actually succeeded
so I think again today today I'm
positive today these are all good things
for me right and the cost really matters
because if you're using a reasoning
Model A lot of that can happen in
inference versus in the training which
is I think less expensive uh before we
move on from this I just want to talk
quickly about this AGI thing that we
talk about so often but rarely Define
and rarely talk about in context right
that all these labs are trying to push
toward uh artificial general
intelligence or human level
intelligence and it seems like some
inside these these organizations are
like uh full-fledged trying to get there
uh and others I don't know probably like
see it as useful marketing so they can
sell products today actually I think
that a lot of the productization that
happened has kind of been an accident in
places like open AI as they've pushed
you know the research forward but why
don't we just take this point in time to
just talk a little bit about AGI do you
think a it all along has just been this
marketing term and do you think that if
we're not going to get there through
these current methods that the magic of
that marketing Falls away a little bit
making it harder to sell into companies
making it harder to fundraise if all
these companies are doing are just sort
of productizing what they have today and
I guess B do you think we'll get there I
think I'm I'm going to go with a and
it's because I guess how would you
define AGI or artif artificial general
intelligence I think Yan lon's
definition is really good which is
basically that it's human level
intelligence it can uh handle a variety
of things just the way that a human
can what is human level intelligence
though because there's a I don't know I
I think chat PT can already do a lot of
things better than but it's it's almost
like yes it can answer you know
questions about philosophy the way a
philosophy professor could but it's
almost like the little more Nimble
things that it really struggles at like
Chad chpt you can't tell Chad chpt to
like you know go uh you know write a
bunch of emails to people you need to
communicate and it does it for you well
it's not really able to do that it's not
really able to switch very well between
tasks it's not very well it's never
really able to you know learn in the
context and get things right the next
time these are all things that I think
make human intelligence special is sort
of the adaptability and the ability to
be as we say General and I don't think
AI is there yet okay that's that's a
fair definition and using that
definition I actually think it's fine
for the industry to not be on the
direction of getting there because even
what you said writing a bunch of emails
to different people I think that problem
could and should be solved soon in
really targeted manners like you know
take your entire existing email history
train something on that use that to
generate new emails and build like a
process or workflow where you actually
validate them like I mean really
practically I think solving that problem
could be possible pretty soon and it's
just not getting solved because we're
still all trying to chase the dreams of
AGI and I I think for me and what
exactly it is the human level reasoning
makes some sense but it's amazing to me
that it it's always brought up
but there's not one like clear accepted
definition or one Clear Vision that's
communicated by the biggest people in
the field the Sam mman and everything
else so it it remains this murky kind of
like dystopian robots taking over who
knows what it is will be a line item in
a contract with Microsoft's investment
in order to like change the profit
structure I mean it's such a nebulous
term that that's why I think it does
represent a distraction from progress
and I don't know what it says about my
life that you're like imagine the
strongest form of artificial
intelligence possible what does it do
and I'm like yeah it just writes a bunch
of emails like oh my God imagine a world
where I'm worried about robot takeovers
and you're just trying to go to inbox
zero
here honestly if an AI could get me to
inbo zero it would be a true a true
miracle I I would really believe in the
power of science I would have to get
through 12,000 uh unread
so just a journalist inbox that's how it
goes but um yeah I don't know I mean it
is interesting that they have build AGI
it initially was like an a sort of like
what I was describing human level
intelligence Adept able to generalize
and now I think it's talked about really
in a way that's akin to Super
intelligence something that's smarter
than humans in almost all fields and can
perform things that humans can't
and that's when you hear like the
messaging coming out of open AI that it
can you know um lead scientific
discoveries and these type of things and
it's like okay that's not really General
that's super intelligence and you know I
think that that has led a lot of the
investment and a lot of the hype around
this that will eventually get there um
but it just doesn't seem like it's going
to be through the traditional scaling of
llms I guess that's my point here yeah I
I agree on that that uh and you're like
I don't care that's good I don't care
that's good that's my new philosophy on
scaling laws with llms it's a good
tagline okay last thing about this did
you hear that um Google I think this is
worth watching they have a new
experimental Gemini model it's called
Gemini exp
11114 and do you know about uh chatbot
Arena where they test which is the best
llms is currently sitting at the top of
chatbot Arena and kicking the butt of
chat GPT 40 preview 0 and mini previous
Geminis all the
clouds maybe Google's got it figured out
whatever they're doing there seems to be
working that it's and this by the way
folks this arena is where people compare
responses to different models and pick
the best one and it's been voted on by
6,000 Folks at this point and is at the
very top so it's quite a moment for
Google that I don't want to glance over
and I think we'll probably be coming
back to it when we talk about Google's
uh prowess in the field yeah I think and
for listeners I mean check out chatbot
Arena it's it's honestly a fun thing and
it's it's a blind comparison test so you
don't know you you're given two answers
you select which one you think is better
and then you find out what's the actual
model behind it so it is essentially an
unbiased test um Gemini or I have a
question are you using Gemini in
day-to-day life no I'm still all in on
Claude and I'm looking at chat bot Arina
and and I'm like I am not doing it right
I mean it's interesting cuz maybe these
models can give a better response but
it's also just like it matters like UI
matters I mean this is sort of me
agreeing with your argument that user
interface matters personality matters
usefulness matters even if the model is
smarter but I this is making me think
it's time to give Gemini another shot
how about you I I don't use it very
often I have it bookmarked but I mean
still Claude perplexity chat GP T steady
rotation all for different use cases I
think perplexity is almost one of the
best examples of like UI completely
transforming how nice it is to use um
and by this point I really thought
Gemini should be my entire travel
booking given it's connected to Google
travel Google flights and everything
Google Maps like it should be my
starting point and it still isn't and
may I still think
the answers in the existing form just
are not very good it gets a lot of stuff
wrong um it doesn't answer a lot of
things as well and I get trying to be a
little bit more conservative and risk
averse I still think Google is
incredibly well positioned just given
their ecosystem but it still has not
gotten there yet and I mean it's on
maybe the next experimental model once
it becomes reality will will kind of
cross that Chasm but we're not there yet
yes and I have to say I have become a
bit of a of a perplexity guy I'll admit
it perplexity is pretty good it's so
good it's my it's become my kind of like
companion for other things like I think
chat GPT is more when I have like I'm
sitting and doing something focused CLA
is for a lot of work a lot of like more
on the coding side and kind of like
really the art using artifacts
perplexity is when I'm watching
a movie a sports game of any sort like
like it just is so good in just giving
you quick information in a really nice
format with additional links to keep
exploring in questions that uh I think
for that like and which makes me think
it's almost it and it still is the
biggest competitor to real search yeah I
find it to be really good for research
like if you're trying to get like
imagine trying to sort through a bunch
of programs and figuring out what they
offer or like having like a broad like
eligibility question like all right I'm
traveling to China how do I deal with
the VISA process something like
perplexity is way better than Google
just going to say actually yeah yeah
yeah I I am just making the decision on
the Epic Pass versus the icon pass for
winter skiing if other if listeners are
making the same decision and all done in
perplexity and asking like specific
questions which resorts in Vermont which
resorts in Colorado are there and like
it was so so good in doing that
I do hope that perplexity finds a way to
get me into China uh this winter which
I'm hoping to stop in on the way back
from Australia so finger crossed where
are you looking to go Beijing I want to
see that
wall see that wall I'll report I was
there in 2009 I went to the Great Wall
it it was definitely it was a good time
it was lived up to the billing nice so
okay so let's just take a minute and go
through three stories that the two of us
kind of found this week that are talking
about what you really are interested in
Ronan which is the practical application
of AI at this stage and how even if we
stop right now which I don't think we
will we're going to have an extremely
powerful technology that's going to
disrupt Industries and really be
practically useful so why don't you kick
it off with this ryer fundraising that
you talked about uh right before yes so
ryer is a generative AI startup they
just raised 200 million at a nearly $2
billion valuation what's interesting
about them is they're they built their
own F Foundation models we had just
talked a little earlier about how they
build more kind of targeted models that
are really focused on solving Enterprise
business problems and the entire kind of
differentiation that they're focused on
is kind of exactly what I've been
talking about like they have a lot of
bigname Enterprise customers going in
and like remember these companies have
messy data they have like lots of really
heavy processes that you're not going to
just call and make an open AI API call
and solve like there's so much other
work that needs to be done that I think
it's it represents like more the
Salesforce service now world of
enterprise software versus open AI being
I don't know just like a more pure hard
a pure tech company more like and I
think starting to see more companies
like that that represent the actual
utilization and application layer of
generative AI is going to be a good
thing it's going to be a very good thing
when ryer Le does its job well talk
about like what you could see in helping
a company with uh writer yeah it's uh
sorry wrer look at me I'm adding the Lee
at the end of a startup like we're in
2000 2013 again what a time what a time
um I think what it would look like is
going in and actually taking one large
Enterprise and then
recreating hundreds of existing
processes and just making them better
automating some stuff adding a
generative a AI layer to other stuff
like uh maybe keeping some stuff manual
like like really rethinking every
existing process at a large Enterprise
and then like actually asking how does
generative AI fit into this and then
making that happen and actually creating
the kind of Frameworks and software that
allow you to do that I think if any
compan is going to be able to win on
that that's where the I mean the value
that's going to be accured there is
going to be massive versus just again I
was shocked when I saw open ai's kind of
like Vision around its business is still
chat GPT plus subscriptions cuz I still
believe the companies that crack
Enterprise are going to be the ones that
really ACR value in
this that's pretty cool okay so mine is
a little bit different and that is Che
what AI has done to Che this is just an
example again of like how current AI is
going to change things no matter what
and for those folks uh who don't know
Che is an online education business and
they actually started with textbooks and
then built up like a pretty serious
online education business and when kids
would like want to research stories or
problems they would would they would say
that they were checking it and this is a
Wall Street Journal Story how chat chpt
brought down an online education giant
uh basically saying that instead of
cheing kids are using chat GPT now and
Che stock is down 99% from early 2021
erasing some 14.5 billion of market
value and there are Bond Traders they
have doubts that the company will
continue bringing in enough cash to pay
its debts so even today we're already
seeing this stuff um start to uh really
change uh uh education and I think
that's like the most obvious place but
we've we've already had a few years to
see this run its course and look at what
it's already done with ch and I think
that's like a sign of where things might
go uh with the rest of Industry not that
every other incumbent company is going
to lose 99% of their value I think this
was particularly uh a unique case but it
does kind of show how standard it's
become already in education yeah I think
also seeing that 99% draw down brought
me back to CH was definitely one of
those 2021 extrapolating into the future
the pandemic and like anything with the
words online education in it just
exploded in value so I think on one hand
some of it is related to just that not
being the case anymore but also I think
this is actually a really good example
if you think about it like students are
the ones they are they will take
whatever existing technology there is
and make it work for them they will do
that work and they will figure out how
to answer their homework questions or
maybe write a paper in some cases or
whatever else it is and I think like
this is a perfect example of you know a
space where the user is actually driving
The Innovation themselves because
students like free things or cheap
things that help them do better more
quickly so that's a good that's a good
use case user-driven Innovation I like
that and it's not just students it's
it's ad agencies are starting to use it
as well like talk about like trying to
find the answers to the test uh the Ad
Agency the ad industry is a place where
this happens and this is the last one of
the three stories that we're going to
talk about in terms of the Practical
impact today but again from the Wall
Street Journal AI saves ad agencies a
lot of time they still charge by the
hour and this is a story basically that
ad agencies have been charging by the
hour to clients and now all of a sudden
they have chat GPT that's made them far
more efficient for instance you know if
your job if your job was to write
headlines for a brand now instead of
having to come up with 50 unique ones
maybe you can write five unique ones and
as chat GPT to extrapolate out uh or do
the same thing with creative like
creative resizing is now becoming much
easier with generative Ai and all those
hours that Ad Agency spent doing that
work which was really repetitive and not
value ad um now has become pretty
automatable with artificial intelligence
and they're trying to find a new way to
charge and some are going to charge now
based off of specific results versus
hours and maybe we see that in places
like law right and and other disciplines
um so ran I'm curious do you think that
ad agen should still charge by the hour
given that they were probably not
charging for you know such valuable work
a lot of the time and now that chat GPT
has all of a sudden made them efficient
they realized that like a lot of the
things they were doing weren't really
additive to the client I mean what do
you think the best solution is and what
do you think the story tells us I think
first it's incredible that you just
Associated Innovation and ad agencies I
think every Ad Agency out there would be
ecstatic that someone I know we have
advertising listeners shout out to the
ad listeners yeah I uh um I think this
story is much much bigger than just ad
agencies and I I I loved this one
because I think the pricing of
everything that we got used to could
change you just said it whether it's ad
agencies law firms will be very similar
like outcome based pricing and in
healthcare this has been a conversation
for years years the idea of outcome
based pricing where like the actual
results are where you Bill rather than
the treatment itself is a much much
better way to potentially approach this
so I think for so many of these
industries the way the entire pricing
structure changes it's going to change
and I I even think in SAS that's going
to be the case and there's been a lot of
talk around this with even Salesforce AI
agents or and many others is that
seat-based pricing doesn't make sense in
a lot of ways like if you're automating
a bunch of
workflows how many people are using it
is totally irrelevant so that there's
definitely going to be some new pricing
structure that's something around
outcomes around the amount of compute
that is consumed like like it's it's
actually kind of exciting again like on
the productization side of this like
it's going to completely change the way
and different Industries price and it'll
be better I think yeah okay so after
after me like sticking up a whole fight
about the Practical applications of this
technology at the beginning of the show
I'm starting to see it your way I do
think that like there's a lot of room
ahead in terms of whatever we have today
to apply it practically and I think
maybe we should have flipped this like
that's actually the big story and where
the models go is sort of um now that I'm
talking about it out loud I still care
more about the where the models go
actually no are you saying it's time to
build
it's time to build Ron it's time to
build time to build it's time to break
through that ASM toote man and just get
going just break the ASM toote man so on
Monday I spoke with Gustav Sodom of
spotify and managed to fit in a question
about parent mode but we also spoke a
lot about um whether generative AI will
replace music and whether that is
something that can touch someone's heart
whether it was uh developed by a human
or a machine machine and um you know I'm
looking through our deck and I see that
you have uh inserted that story back
into the conversation and uh I'm ready
to hear your reaction to What happened
on Monday all right so first of all and
I had asked Alex to ask the Spotify CTO
CPO about parent mode and my problem is
since I've had kids my Discover weekly
has been destroyed I had screenshotted
my most recent discover weekly when I
opened it and the first song is the poop
poop poop
song yes and basically everything in
there is uh is just something number one
hit it's it's kind of a banger but uh
basically like not being able to
separate out what my kid is listening to
versus what I'm listening to makes it it
just destroys the algorithm and there's
no like I want to hit a refresh button
he had made an interesting comment that
uh like well making different profiles
is actually really bulky and switching
it's kind of a pain I could make
playlists for my kid and then say do not
add these to the algorithm but I think
it's like a reminder that the most
complex Advanced recommendation system
in the world with basic UI problems does
not work and this is another I I think
that's another good example of that that
like you could have and I've read stuff
over the years Spotify how they populate
discover weekly and they're very early
to machine learning recommendation but a
simple UI problem makes it so I end up
with the poop poop poop song as number
one yeah and I do think that they're
they're it's interesting that they're
going to look at these signals and try
to like get better at figuring out
where uh your listening doesn't match
what you usually listen to and try to
exclude that um but I don't know it
seems like a problem that's going to
take some time for sure so enjoy the Poo
po poop song and wheels on the bus I was
like we we were talking about it on
LinkedIn I was like oh enjoy wheels in
the bus you're like no it's much worse
than that it's much worse but actually
did you maybe what could solve it did
you try the new AI generated uh playlist
feature I think no but that's pretty
cool so talk a little bit about that CU
that gusta and I were talking about that
then it came out this week yeah so I so
it's basically you enter a prompt and
you get a playlist and you get a bunch
of recommended songs and you can kind of
like plus plus plus and choose a bunch
of the songs you'd want and so I
literally was like one of mine was you
are a frat boy in the year 2002 in
Atlanta who wants some party songs and
it literally recreated my early colle my
college experience and it was so good
like it really got just the most cheesy
stuff from that period Then I made a
running playlist and I gave it a couple
of examples and again it nailed it so I
it had me start to start thinking like
imagine if it really can get it to where
you just depending at that moment are in
a particular mood and a really really
really specific mood and you just tell
the system that and it creates this
playlist for you and I I think this is
going to be big for them because I think
not everyone is the kind of music
listener which I am like who spends time
making playlists so this could really
solve this problem for a lot of people
yeah and this is what I was trying to
speak with gustaff about it's like what
if you write your prompted and you
actually get AI generated music that
will speak to you more than the human
generated music and you actually also
dropped this in our document I was like
what am I looking at here and it's a
bunch of drone video with this like
really lovely song in the background and
the song you later let on totally AI
generated yeah I I got a drone recently
and have been having some fun making
some with it and I was up in Hudson
Valley in a town called Cold Spring New
York and literally just with sunno made
a prompt uh it was like you write a song
in a folksy acoustic style about a town
named Cold Spring in Hudson Valley and
talk about the foliage so I made this
video put this song as the backing music
shared it with my family in a apple
photos shared album that we use and my
uncle was like this is a beautiful
singer who is she and then there was
that moment of I'm like do I divulge and
then I did I was like yeah it's AI which
which blew some minds I think the song
was genuinely good yes it was really I
enjoyed it very much all right and also
now that we're talking about Spotify
I'll just note that we are now doing our
Wednesday shows uh via video on Spotify
so if you've recently found the show
this is how it goes we do Wednesday
interviews with folks in the tech
industry or Outsiders trying to change
it
and then on Fridays Ronan and I talk
through the news so these shows will be
Audio Only across all platforms the
Wednesday shows video on Spotify and if
you're new here we appreciate you coming
aboard and giving the show a shot
definitely seeing a bunch of new
subscribers uh come in and we appreciate
you all uh before we go to the break
just want to say uh share some gratitude
to a couple of our listeners uh first of
all context
1930 um shared a comment on Trump in the
reviews uh of the podcast and it was a
critical review but it was five stars
we're taking it into account and we
appreciate the way that you shared that
feedback it helps us and it helps the
podcast and I think that's uh the best
way to do it so thank you context 1930
also Luke Squire uh made a comment about
our discussion about polling versus
prediction markets on LinkedIn basically
in favor of polling versus the
prediction markets we got a couple of
those uh and that's another great way to
to share feedback and and thoughts on
the show is shared on LinkedIn and
critical or not we love to hear what you
think about the show and it obviously
gets the word out to others so we
appreciate that thank you Luke and then
uh Graham High uh emailed me with our on
our uh email address uh for feedback
which you can find in the show notes and
uh made a very interesting point so we
talked a couple weeks about how the
government should build its own starlink
we talked about that a few weeks ago and
Graham pointed out and I'm embarrassed
to admit that didn't know this uh that
the Department of Defense has actually
already uh started work on its own
satellite internet company or or
Communication System working with SpaceX
it's called star Shield Ronan did you
know about it here's from uh one story
about it it's a militarized version of
spacex's starlink Internet Satellites
with enhanced encryption and other
security features and unlike starlink
which is a commercial service the star
Shield satellites would be owned and
controlled by the US government the
government is actually building this I
did not know that but I think we need to
do more space coverage that's right I
think uh I think space for 2025 is going
to be a a good topic all right Bezos put
us in a spaceship get we'll take our
mics and we'll do it and uh live from
Blue origin that's right 2025 goals y
Bezos we know you listen so just do it
all right let's take a break we're going
to talk about blue sky and if we have
time we're going to talk about
um Apple smart glasses right after this
and we're back here on big technology
podcast Friday edition just a few
minutes left but I definitely want to
talk quickly about this blue sky surge
uh so blue sky is now up to 15 million
users and it is uh it's it's really
soaring in the wake of the election I
don't know about you but I've definitely
noticed myself and lots of other folks
have talked about how they've seen Mass
amounts of followers delete their
Twitter accounts and
I think blue sky and threads which
threads has added 15 million users in
just uh since the start of November have
definitely benefited from this so do you
think that this is has staying power or
is it a flash in the pan I think it does
have staying power this time so I went
back to the Blue Sky account I'd created
like a year and a half ago maybe and it
was interesting I actually saw people
who I would engage with with on Twitter
all the time who I hadn't really
processed had left but just kind of
hadn't thought about or noticed in a
while and suddenly it was like oh wait
they're they're Alive and Kicking and
just having those same conversations
especially around a lot of like
economics topics Finance topics even in
Tech as well I found a lot of the a lot
of tweeters from my past uh in there so
I think because again from a product
standpoint from before even like
creating an account signing in following
was kind of a pain
and then now when I went back it's it's
pretty much on par with Twitter SLX and
so I think there's there's staying power
here because again the actual technology
behind any of these apps is not that
complicated it's purely about the
content and the people involved so I
think it does represent a risk this time
but we've said this a few times now so
yeah and I'm about to pour some cold
water on this uh Max Reed who writes
Reed Max on sub he says uh from what I
can tell the users who've been joining
blue sky and mass recently are members
of The Big Blob of liberal to left-wing
journalists academics and lawyers and
Tech workers uh politically engaged
email job types who were the early
Twitter adopters and whose compulsive
use of the site over the years was an
important force in shaping its culture
and Norms but he says uh blue sky is
really acting more like a large Discord
server a place to socialize
banter and kill time than a proper
Twitter replacement so basically what
he's saying is it's inhabited only by
those people and it feels a lot like the
old Twitter but it just doesn't have the
user numbers that it used to have and
therefore the Blue Sky boom uh might be
an illusion what do you think about that
no so I think when I had gone on it way
back it was like the extreme version of
the anti- Elon Musk uh anti- Twitter
types this time there's a lot of sports
highlights on there which could be my uh
I mean there there there's more kind of
Normy content on it this time around um
and I pretty quickly was able to find a
lot of good follows so I think that it
that's still kind of how things were and
maybe that's his specific feed but I
think it's different this time let's see
I don't think that it's going to work
one thing we can say say for sure is
doesn't look like threads is working I
mean threads added 15 million people
since um the start of the month and uh
Michael lion who I work with as an
editor pointed out to me he's like does
it feel like that no feels like the same
thing it's just people complaining about
threads I can't with threads I I I
opened it up again and yeah I mean it's
so odd in terms of like and I tried to
follow a bunch of people on it but I
don't know it's it it just does not
deliver kind of more real time
interesting conversation I will admit
like blue sky I actually moved it to my
homepage on my iPhone and moved X off of
it and then in New York this week on
Thursday we were looking out our window
and you saw smoke coming out of a
building in Midtown I did you even hear
it yes of course so there was a fire in
Hudson yards apparently it was like a
mechanical room blew up or something
like that but uh and no one was injured
or anything but but I I actually tested
I went to Blue Sky and searched NYC fire
nothing I went to threads nothing I went
to Twitter and got all the info I needed
right away yeah that's what I think
Twitter is going to be the one with
staying power just the network effects
it's very very it might be the most
difficult to replace social network like
we've seen like blue Facebook at least
in the US start to lose a lot of
interest people are on Instagram now I
just don't see it happening with Twitter
because it is just the group of sickos
that have been on that platform and the
network effects there is very difficult
to displace okay last story of the day
apple is thinking about smart glasses
was in our Dock and for like the last
week but there's been a lot of politics
to talk about this is from Bloomberg
apple is exploring a push into smart
glasses with an internal study of
products currently on the market um the
initiative code code named Atlas got
underway last week and involves
Gathering feedback from Apple employees
on smart glasses uh and it's been led by
Apple's product systems quality Team
part of the hardware engineering
division so it's very interesting to me
that smart classes are already becoming
a thing meta has a great pair out with
the rayb bands and Apple has been beaten
to the punch here and I think uh it's
not going to be too long until we see
apple build a product like this of their
own if not one with an enhanced Siri to
hit one of your most favorite things
what do you think yeah I think I I mean
I told you a couple weeks ago that I'm
testing the Snapchat the snap spectacles
which is their new augmented reality
glasses that you can get as a developer
versus Orion from Facebook the actual AR
glasses are not available for any kind
of like General release but after using
the smart the
spectacles it's they're amazing and and
I talked about it like even my son can
use them instantly my mother like anyone
of all ages and kind of like Tech
technological proclivity can just pick
them up and use them and I think this is
the this is the form factor of the
future this is like what we're all we
are going to all own some kind of
glasses and apples got to get on there
quickly and The Vision Pro was not that
and VR
yeah what do you think it says about
Apple that they haven't been able to do
this it's not a good sign no it's not a
good sign I think like apple
intelligence I mean it's if you think
about it we have a bunch of Misses in a
row Apple intelligence maybe it'll come
around but it is so far from anything we
have seen even remotely close to useful
The Vision Pro flop I mean and I'm still
upgrading my Mac and airpods and iPhones
and all that but it's just it's I mean
and again at their scale they need to
find that next big winner we everyone
knows it and it does not feel maybe Siri
will work in a few I think Apple's best
chance is that Mark Zuckerberg gets so
ahead of himself on his rebranding
campaign where now he's like you know
cool MMA Zuck with the chain and the big
t-shirt and the long hair that he
distracts from the mission
and then gives Apple an opening and
uh you know I'm a fan of a lot of
Zuckerberg's side projects but there was
one this week that I just didn't think
hit and raised some Emoji red flags for
me and that was a collaboration that he
did with
Tay uh to sing a song uh low it's called
Low it's a one of the songs back in the
day it's G low Alex it might have made
it into my Spotify AI playlist from
20000s uh college party music and he
worked with uh Tay to record a version
of this song it's quite x-rated and uh
Tain wasn't even involved in get low
back in the day but this is from
Business Insider the duo which calls
itself Zain released the Slowdown not
safe for work track on Spotify and and
it features a heavily autotune
Zuckerberg syncing original lyrics about
going to a club and getting confronted
by a security guard and it features
Zuckerberg singing some lyrics that I
really never wanted to hear him
sing hey are you there yeah yeah oh
sorry I wasn't sure if you're pausing to
leave room for the okay yeah I'll just
say it it it features Zuckerberg singing
some lyrics that I never wanted to hear
him sing uh and what would those lyrics
be you may know it from the window to
the wall and then I'm done I'm not even
going to say then there is some sweat
ding there is there is some sweat we're
not we're not going to jump the shark uh
like Zuck did by doing this song but we
will yeah go ahead oh this was awful I
mean what I kind of love is is thinking
about like trillions of dollars of
market capitalization potentially
swinging on Mark Zuckerberg sitting down
with tay with an acoustic guitar I can't
does he actually play it in the video or
but singing I'm trauma wiping it from my
head yeah and singing about sweat in the
nether regions and with and to me the
most ridiculous part is that as you said
in the Business Insider article said T
Pain didn't even sing G low it was Lil
John and the East Side boys back in the
day so like just how this came to be and
what this could mean like you're you're
going around laying off people telling
them this is the year of efficiency and
then you're trying
to call yourself Z pain and come up with
some weird alter ego and SW oh man I I
can't with this one this one was too
much Ronan I think there's only one
thing that's left to do at this point
what's that uh that is to uh queue up
the song and play about as much of it as
we can get away with without being
kicked off of the podcast platforms so
thank you for coming on the show ranan
thank you everybody for listening and
now to play us out zpay Mark Zuckerberg
and tpay we'll see you next time on big
technology podcast