I work on the intersection of AI, Software Engineering and Programming Languages. Currently, I am applying Formal Methods to Agents, towards AI Safety.
While work is obviously a very, very complicated thing, a useful lens for the purpose of this essay is to draw a distinction between work which reshapes a raw material into a finished object and work that puts together multiple objects in a way that creates a certain effect in the world. For the sake of having a shorthand, I’ve chosen to call them transformative and compositional work, respectively.
While Iris does not have a particular clear idea of LLMs, Jeremy Keith does:
My own take on this is that transformative work is often the drudge work—take this data dump and convert it to some other format; take this mock-up and make a disposable prototype. I want my tools to help me with that.
But compositional work that relies on judgement, taste, and choice? Not only would I not use a large language model for that, it’s exactly the kind of work that I don’t want to automate away.
Transformative work is done with broad brushstrokes. Compositional work is done with a scalpel.
Personally, I think it depends much more on where you are in the economic value proposition. Are you selling quick-and-dirty cheap stuff? LLMs are great for you. Are you delivering high-assurance, high-quality work? Then you might/should be skeptical of LLMs.
This protest highlights the tension between creating valuable tools and devaluing human content.
The value of AI
ChatGPT (and even Apple Intelligence) is trained on information publicly available on the internet, data from (consenting) third parties, and information provided by their employees or contractors. Over the last year and a half, people have been amazed at what ChatGPT has been able to do. Although the quality of its work fluctuates as new data/methods are being updated, ChatGPT and similar tools are being used to create value. But at what cost?
Unconsciously, The Algorithm has become more and more important in our lives. From Instagram and TikTok reels, X and Facebook timelines, Spotify, YouTube, or Netflix’s recommendations, the decision of what we see is no longer ours. And we are also not delegating our choices to a human editor (as is the case of the old boring telly or radio channels). Those decisions are being made by black-box algorithms that are hidden in the shadows.
The EU AI law, which I blogged about before, only requires explainability for applications in high-risk domains. Entertainment can hardly be thought of as high-risk. However, I would argue that given the importance of online content consumption in today’s society, it should be considered high-risk. One example is the perceived power of Twitter/X in political elections.
On the other hand, educational purposes are considered fair use in most countries (which is certainly true here in Portugal). What is the difference between fair use for human and machine learning? As we become increasingly dependent on AI for our daily tasks – I use Siri and Reminders to augment my memory and recalling ability — we become de facto cyborgs. Is there a difference between human and machine learning for education?
The devalue of Human content
In 2017, Spotify introduced the Perfect Fit Content program, encouraging editors to include songs purposely designed to fit a given mood in their playlists. Liz Pelly goes into all the details in her piece The Ghosts in the Machine. Some human, some AI, several companies have been starting to produce music à lá carte for Spotify.
According to The Dark Side of Spotify, Spotify investors are also investing in these companies (phantom artists on the platform, which use random names with no online presence other than inside the platforms) and promoting the use of AI to beat the algorithm. While this vertical integration might be cause for considering anti-trust or monopoly issues, the fact is that Netflix has been successful in expanding to content production (as has Disney been successful in expanding into content distribution).
AIs are much more productive in generating music than humans. Which is not necessarily the same as being successful in producing music a) that humans enjoy or b) that is commercially viable. The Musical Turing Test is almost solved, addressing a). Commercial viability is even easier to address. Because the cost of producing AI music is so low compared to the human equivalent, AI companies can flood the market with millions of songs, letting the algorithm filter out the ones that do not work. In that scenario, human musicians are not just competing with each other for user’s attention but are now unable to be showcased to users without an explicit search. Additionally, AI can better cater to some audiences based on data extracted from these networks (remember Spotify’s investors also investing in AI music production companies?) than humans can, at least in large numbers.
And I’m aware AI can be a tool for musicians, but if AI can perform end-to-end music generation passing the Musical Turing Test, it becomes much more interesting from a commercial standpoint.
The only chance for musicians is to promote their own content outside of these platforms, abandoning the initial goal of Web 2.0, where anyone can create content on the web. They can, but it just won’t be discoverable in the ocean of AI-generated content. But this is a symptom of a more significant problem for the web.
I feel like the people who try to be positive – well, I don’t know what they’re doing. I’m a music producer and also a writer who also happens to make art/design her own album art. Thankfully, I also dance, which is going to be the one thing that saves me I feel. — PrettyLittleHateMaschine on AI music.
The quality of AIs depends on human
ChatGPT was primarily trained on internet-available content. So, its quality depends on what is available at a given time. If we stop collecting new information, we can assume its quality will remain unchanged. Still, it will not be helpful with new information, such as news updates or scientific discoveries. Its usefulness will be reduced.
On the other hand, if the quality of AIs increases — it’s more and more difficult to tell the difference between human and GPT-generated text — and it passes the Turing test, the content available online will be more and more AI-generated than human-generated, as it’s more economical to use AI to produce text, audio or even video.
Here, we consider what may happen to GPT-{n} once LLMs contribute much of the text found online. We find that indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear.
This recent Nature paper reports that LLMs perform worse when trained on LLM-generated content. Human content is now essential! LLM companies need high-quality human content to train their next-generation models, especially concerning novel knowledge. But econmics no longer work. Content is created once, consumed once, and used to generate millions of derivates for almost free. An author might publish a book, hoping to make the money for the time it took to write from the sum of all individual sales. However, AI companies will not buy the book at its production cost to train a model. Same for daily news. The human audience is still needed to make this work. And suppose everything is made available for free on the web. In that case, humans are making the same mistake that led to ChatGPT being in business without contributing to the original content sources.
The current Web is not enough.
Web 2.0 died and now the web happens more and more inside silos. Famously, Instragram does not allow for links outside its app. “Link in the bio” will be listed as the cause of death in Tim Berners Lee’s obituary. It goes against what the web was supposed to be. But today’s personal entertainment happens in silos (Instagram, Netflix, Disney+, etc…), not on the open web. Even Reddit communities have started blocking links to some websites, like X.
The web failed at microtransactions. Paying 10 cents for reading a well-written article was the original goal. Even with Paypal and Apple Pay, the model was only successful for large purchases, not pay-per-view. Imagine that you give Youtube your credit card, and it takes 1 euro for each hour watched. Once you have something for free, it is difficult for companies to make you pay for it.
As a business that moved from analog to digital almost completely, most news outlets have failed to change their economics and they are now struggeling financially. As the price of advertising online has decreased over the past years, they have switched to a subscription model, putting up paywalls with dubious outcomes.
The future of the Web
I foresee a web where high-quality human content is behind paywalls. While most of the web can be AI-generated and free, it will be ignored if high-quality content is available from trusted sources. Content will be signed and (possibly) encrypted using personal keys. These keys can be provided by the government, or other parties. For instance, every Portuguese citizen already has their keys inside our citizen cards, sometimes with professional attributes.
If you wanted to read the news, you can go to an online newspaper, where the content will be signed by a recognized journalist or editor. The body of the text can be encrypted but with a faster Apple Pay-like prompt, you can pay cents to read it. Even if the journalist published AI-generated content, they are liable for its content.
This proposal makes the web a more trustful place and somewhat addresses the economic problems of paying for content on the web. It requires payment processors to drop the minimum cost per transaction, which I believe is happening more and more. And as more and more garbage is published online, users will see the need to pay for high-quality content.
As for AI providers, they will now have to pay for content. And even if it is ridiculously cheap, there is a trace that they bought that information, useful when you want to prove in court that your content was used in training LLMs.
We might not get to this Web, but I hope some of this ideas help the web survide the tsunami of garbage content that is starting to flood our dear World Wide Web.
The reason for the ban is homeland security: Chinese entities can use TikTok usage data and content to gain access to private data on US citizens, as well as use personal content for blackmailing and influencing the public perspective. They do have a point about security, but should we also ban Facebook, Instagram, Google and others for their possible control of Europeans? We need to fight for our technological independence.
I don’t mean to sound like an old man, but this is becoming an issue of Public Health. And if you don’t want your government to shut down “addictive apps”, we must self-regulate its use first.
Think of the case, which is quite common, of a humble head of a family who injured his foot with a piece of glass, a stone, a tack, or in some other way, and who, as a result, contracted an infection that resulted in his death. Evaluate how that family will live, whose only support has disappeared thanks to lack of foresight, which is the result of a bad habit. (Government ad to promote the use of shoes, translated from an old style of Portuguese)
More recently, the Covid curfews implemented in many countries were a similar measure: taking away people’s freedom allowed the health of the population to improve. And these are not democratic decisions. They are mostly based on a technical analysis.
Now, there are technical reports that TikTok is affecting Public Health. It is the first time the new generation has more difficulties learning than the previous one. Mental health is at its worst and we need to act upon it.
Conclusion
I don’t have TikTok installed. I don’t watch reels or shorts. I try to watch and read long-form content. I teach my students that you need a larger-than-average attention span as a programmer. You should train it. And we should encourage the next generation to appreciate long-form.
I’ve been following Gwern.net for a long time, and I was really curious when I found out this interview that kept him anonymous.
How does an hearing-impaired, introvert American grow up to be a polymath and a scholar?
The wikipedia editor, now full-time writer without a salary
got a lot farther by writing a lot and harder,
by being a lot smarter, by going down rabbitrabbitholesholes
by fourteen, they placed him in charge of wikipedia edits
🎶
And it’s a bold move on LG’s part, considering most folks just want to see their family photos or some calming art while they’re not actively binge-watching. Even if you can turn it off, the default setting is a bit of a slap in the face for anyone who thought they’d bought a premium product free from such annoyances.
This shift towards monetizing every idle moment on your TV is a slippery slope. It’s not just about selling hardware anymore; it’s about squeezing every last cent from customers, and brokering the data to get more revenue. And while LG claims this will boost brand awareness, one has to wonder if viewers will just tune out entirely (or worse, switch to a platform that respects their downtime). As the lines blur between content and advertising, it feels like we’re all just one step closer to a world where even our screen savers are working overtime.
Someone should send a copy of Black Mirror’s Fifteen Million Merits down to LG leadership. As long as profit is all thesecorporations care about, and as long as the users don’t stand up to this, it’s all going downhill from here.
Because I’ve always seen reading time as such an incredibly personal thing, I’ve never once paid any heed to these widgets. In fact, I’ve been slightly perturbed a service would presume to know how quickly I could read an article.
I’ve always viewed any service that sticks a “reading time” widget on its articles as the literary equivalent of fast-food: you’re not here for quality, but for expediency.
Personally, I think they devalue a text more than they add to it.
I’m with Nielsen on this one. It makes no sense to suggest an (average) estimated reading time, unless you are trying to convince your audience that this is not a long piece, and you can read it in a minute or two. For personal blogs like this one, I really don’t care if people read it or not. But if you are writing one of those linked-in-self-promotion-bullshit-crypto-AI-latest-fad types of posts, then your goal is to get the reader to take its time. And in those cases, you can bluntly lie and always say it’s one minute.
My guess is that this trend comes in response to instagram/reels/shorts/tiktoks where people do not want to spend any more time on a particular content than they need. Long-form reading is dying, especially on the internet.
Other than teaching how debilitating this is to the younger generation, I hold a very pessimistic view for the next generation.
Starting with this build, we are introducing the Power Grid Forecast API. This API empowers app developers to optimize app behavior, minimizing environmental impact by shifting background tasks to times when more renewable energy is available in the local electrical grid.
— Announcing Windows 11 Insider Preview Build 26052 (via Terence Eden)
Guilherme, Paulo and I submitted a proposal of a similar service (but for server workloads, where it makes more sense) back in 2020 for an EDP competition. It was not fancy enough as it was mostly transparent (the API was a job queue).
I remember the dark old times where IE6 was the default browser everywhere. Because of that dominance, developers focused solely on the IE substandard of JS/ActiveX/JavaApplet, leaving Linux and Mac users behind. Not long ago, you were required IE to submit your taxes. That has changed: Now you are required Chrome or Edge. As a Safari user, I find myself having to have an installed Chrome to access unsupported websites.
This week, MS announced they would be dropping their own rendered (EdgeHTML, a fork of Trident) on their current browser Edge and they will be using Chrome’s Blink (itself a fork of Webkit). So now the major browsers are:
Edge (Webkit/Blink)
Chrome (Webkit/Blink)
Firefox (Quantum Render, replacing Gecko)
Safari (Webkit)
Except for the tiny differences between vanilla Webkit and Blink, almost all the web uses the same renderer. This is the same monopoly Trident (IE5/6) had more than 10 years ago! And the sole fighter for a diverse web is the same browser and team that fought then: Mozilla Firefox! And this is not by chance: Mozilla’s Foundation is all about diversity and open standards. Just check their funded research projects to see that they put their money where their mouth is.
If it weren’t for Firefox, I believe Chrome would never had the success it had. Now Chrome’s the one that’s monopolising the web, and we need Firefox to be an alternative that will allow the NextBrowser™ to replace Chrome in 10 years.
I’ve been using Firefox as my main browser for a while now, and I can heartily recommend it. You should try it (and maybe talk to your relatives about it at Christmas). At this point, which browser you use no longer feels like it’s just about personal choice—it feels part of something bigger; it’s about the shape of the web we want.
If you aren’t aware of pixels.camp, it’s a 3 days hackathon in Lisbon, Portugal. This time I’m living 3 metro stops away, so despite not taking days off, I’ll be participating.
One of the bigger changes from previous editions is that a new voting system will be used, based on EXP, a closed crypto-currency on top of the ethereum.
Basic rules:
If you are at the event, you get badges, each amounting to 100 EXP (I’m ignoring the variance here).
There are angels who own a lot of EXP’s (25.000).
There is one lucky guy who may win 50.000 EXP by solving a scavenger hunt.
Participants can create projects.
Everyone can invest in projects. The invested money is controlled by the organization.
At the end of the event (and presentations), all the money invested in all the projects is redistributed proportionally to the 10 “richest” projects. These are the main winners of the event.
After this, the EXP collected by each project are distributed by the investors proportionally to their investment.
I believe this approach is flawed.
Problem 1: Money is worthless at the end.
Unless you are one of top 10 (out of ~100) project owners or one of the X top investors, you get nothing. There is no actual interest in saving your money, or betting in low-risk investments. You want to hit the jackpot by investing in the number one project, if you are looking for winning the investor prize (I’m assuming the angels are out of the race).
Additionally, if you are the scavenge hunt winner, you should be extra careful with how you spend your money, not to lose it (because that money without investing is useless).
So, you should invest in the best company. The question is when.
Problem 2: Project presentations are in the end of the event.
There are no “investment” rounds here with no incentive for investing early. The best approach is to invest in the end, where you already have an idea of which projects have a chance of winning. What happens if everyone invests in the end? Maybe that’s the most fair scenario, but it is no different than a regular voting system without decentralized coins.
Problem 3: If you are competitive, you should invest in your own ideas.
If you really want to win this, it makes sense for you to invest early in your own project. If everybody does this, then the prize is only decided by the angels. Except if you do the right strategy:
Get a small team of developers who are interested in doing something useful. Alternatively, create vaporware. Doesn’t matter as long as the angels are interested.
Get a large team of miners, participants who are not interested in building something, but rather want to collect badges and participate in activities.
This will be the best approach towards winning (unless you have the best project according to the angel’s opinions).
Problem 4: There are no rewards in this Kickstarter.
The project presentation is supposed to work like Kickstarter. The most successful projects on Kickstarter had excellent rewards for investors. Here, unless you are rich, you have no real rewards.
If I end up creating a service as a project, I will give pre-access to the service to whoever backs me. If I am doing a prototype of something that will not last, there is no real reward I can give out (except free hugs).
Problem 4: Work vs Play
You should focus either on working (to get one of the 10 prizes) or playing (and getting badges to be a great investor). I don’t believe a half-way approach is very useful towards winning prizes (Although it might be the best fun experience).
Idea 1: VC-like groups.
Let’s say I invite everyone I know to a VC company. We decide amongst ourselves in which project we want to vote. Only the most voted project gets all the money from the group members. This might seem unfair for members who didn’t vote for it, but in the end it gives that project a better chance of winning, thus making VC members richer.
Btw, contact me in case you want to join Alcides&Friends Ventures, LLC. The larger the VC group, the larges the changes of winning a prize.
Idea 2: Betting
If you have been to previous editions (called Codebits at the time), you might remember the launch of a closed instance of Meo Wallet. It was the same idea, although projects didn’t ran on this. Rui and I won the prize for most money transferred, and came close second for the most money totaled (if only I hadn’t been so generous).
Idea 3: Ponzi scheme.
This was something we also worked on. I’ve known Ponzi schemes to last for months, and we only need it to last 3 days. So contact me as soon as you can to be one of the first to take advantage of the Alcides scheme™.
Don’t get me wrong, this is an interesting idea and we should use these events to play with ideas like this. But I have my doubts it will work in the end like it is intended to.
Source for first website – table based layout, a lot of view source, a lot of Notepad, a lot of IE 6. Used to work mostly in HTML and CSS. With the help from books like “HTML for the World Wide Web – Visual Quickstart Guide”, learned a lot as a tinkerer.
Two years in: good with HTML (table layouts) and moderate CSS (fairly new), basic PHP, could use FTP and do basic web config. Could get a site up and running from scratch. This was enough to get my first developer job. This was without any computer science background.
Now: front end developer with 10 years experience, not an engineer, or a code ninja. I don’t know Angular, React, WebPack. I don’t even know JavaScript inside out. I am valuable to my team. Need more: empathy, honesty, being able to see stuff from a user’s perspective.
Back in my day, we learnt how to do things. Nowadays, kids learn how use high-level APIs, without any idea how things work underneath. They might learn Meteor, but have no idea about HTTP or Sockets or how HTTP Sessions are implemented. Which is fine for developing tiny little apps, but they miss the I understand all this sh*t feeling.
Supposedly high-level frameworks allow developers to write more complex programs in the same timeframe. However, I don’t believe this is true for small projects, because the setup time is increasing exponentially. Let’s start a new single page app, what do we need? Node, npm, webpack, angular or react or any other trendy framework. Say what you will about PHP, but it was a single one-click WAMP install away from your fingertips.
If you were a 13 year old kid wanting to develop your own app, what would you use?
After somesteps to embrace the OpenSource model, specially thanks to IronRuby and IronPython projects, the day has come.
Microsoft is shippingOpenSource tools as part of one of their products: jQuery will be part in ASP.NET MVC and Visual Studio, with Intellisense support!
This is great news not because of jQuery itself (nevertheless, my congratulations to John Resig’s team), but because Microsoft is selling a product together with OpenSource code. This has been battled with a lot of effort by the IronPython and Ruby teams. For instance, IronPython is OpenSource, but cannot accept contributions from the community (in source code, bug reports are welcome). And until today, I thought they were doing the same approach with the JS toolkit for ASP.NET.
There’s this project GimmeECMAScript (or Javascript if you prefer) library designed to make working with “everyone’s favorite scripting language” fun again!_ It is OpenSource, but since it was made by Microsoft:
Due to some licensing restrictions, code contributions from the community will not be accepted, however the Gimme source code is completely free and open to all who wish to view it and learn from it.
I’m glad Scottgu decided not to go with Gimme but with jQuery. (nothing against Gimme, but the community around jQuery is so much wider) This is the real step that tell us that Microsoft is really changing!
Qual a maior lacuna no panorama dos Social Media portugueses?
Paulo Querido: “A falta de participação.
A total ausência de informação nos mainstream media.
A fraquíssima prestação do país em termos de empreendedorismo. Os projectos portugueses que existem nesta área são TODOS de indivíduos ou pequenas startups, sem capacidade financeira. Não há mercado nem investimento nem apoio de nenhuma espécie.”
