The situation in academia is dire. More students are failing than ever and, worse, others are passing without being able to apply the expected skills the following semester, only to find themselves unable to get a job after graduation. Classes are empty, morale is low, socialization is at its lowest, and no one knows what to do about AI.

Both MIT and, closer to our local reality, Coimbra¹ have released reports on AI in education and offered recommendations for action. Here, I present the recommendations I agree with, the one I do not, and a short list of the actions I consider most important.

Worrying Data Points

Success rates are decreasing. Each year, the number of failing students increases. Furthermore, grades no longer form a bimodal distribution: I have been awarding more 20s (out of 20) than ever, and more grades below 4 than ever. The difference between the two modes is larger than ever.

Attendance is decreasing. This is happening all over Portugal, but also at universities around the world, including MIT. Here in Lisbon, ever-increasing housing prices and a lack of dormitories prevent students from living closer to campus, leading to two-hour commutes each way.

Students are less social. This might still be a consequence of COVID, but students now socialize via Discord more than they do in person.

Companies are not hiring junior developers. The 100% employment rate for software engineers led many people to believe that a good job was guaranteed. This is no longer the case, and students do not know how to distinguish themselves in a more competitive market.

The Diagnosis

While we recommend against imposing a one-size-fits-all policy on AI use, students are anxious for clarity about AI use: in any given course, they want to feel sure about when, where, and why AI is prohibited, allowed, or required.

— The MIT Report on AI Use in Teaching, Learning, and Research Training

Both the MIT and Coimbra reports identify students' uncertainty about how to apply AI in their education as a concern. I disagree. Students are better than we are at adopting new technology: Google and Stack Overflow were two earlier technologies that changed how software engineering was taught, and students were the first to understand how to take advantage of them. My evidence is that I have never awarded so many top grades as I did this past year.

Instead, I believe that the main problem is motivation. Since primary school, students are trained to see tests as the positive-reinforcement signal. If an activity does not count towards their grade, they simply do not do it. That has led us to allocate 10% of a grade to homework; top universities do the same thing. Students do not care about learning for learning's sake—or practising after they finish school. They care about passing courses because that will give them the degree they expect to earn and, they hope, a good life.

AI has exacerbated the issue. Students can drag and drop an assignment into Claude Code or ChatGPT, wait less than an hour, and receive a solution that scores 100%. Unlike some of my colleagues, I do not think this reflects students not knowing how to use AI to learn. They know that they are cheating themselves out of the opportunity to learn by doing. Engineering degrees have a strong practical component because people learn best by building from scratch, or from pre-approved foundations. For example, you learn more by writing your own compiler than by reading someone else's source code. The journey—finding problems and developing solutions—is what builds knowledge and expertise. Asking AI may give students enough knowledge to memorize for an exam the following day, but not enough for it to remain with them through the end of their degree and into their careers.

So I identify two main challenges: what skills will a software engineer need in three to five years, and how can we set up incentives that help students obtain those skills?

What skills will a software engineer need in three years?

This is the million-dollar question. AI is increasing the pace at which software is developed. Some argue that this increased pace comes at the cost of software quality. I am more positive about AI: I think the latest models and harnesses together produce software of higher quality than the average human team. This happens because those harnesses are built on the shoulders of giants: open-source compilers, linters, verifiers, and testing and validation frameworks.

I have come to the conclusion, over the last year, that the conventional university degree is obsolete. We have tools that can prove hard theorems and write thousands of lines of assembly in seconds. When Claude can produce something quite close to a PhD thesis in less than an hour, what are we even doing?

— Daniel Lemire

While I am not as negative as Daniel, I believe the required skills will be very different. I do not think people will need prompt or agentic engineering: the major labs will make these tools usable without much effort on our part. Instead, the work of software engineers will shift towards requirements, both functional and non-functional. My view is that we will need two types of developers. Product engineers will determine a product's requirements, use LLM-based tools to generate it, and ensure that it meets its quality requirements. These engineers will be better able to deal with ambiguity and people-focused work. Infrastructure and research engineers will work on the tools that product engineers use—LLMs, networks, distributed systems, and so on—and use AI to test new ideas and do more revolutionary work. They will require deeper technical expertise. To reflect this future, we should rethink our curriculum to cover product management, which is typically treated lightly, and the foundations of all major areas of computer science: networks, distributed systems, programming languages, databases, and AI, including neural networks. We also need to teach this last area earlier, rather than reserving it for advanced Master's-level topics.

But this is only my personal opinion. Others may have a different view of the future. The most important conclusion is that we need to become more agile in adapting curricula. We used to revise them every five years, with major revisions only every 10–15 years. We now need to be able to revise a degree from one semester to the next. Things are changing quickly, and the pace will only increase. We need to adapt not only course content, but also how teaching works.

You need to be able to change course and program content quickly. Most universities spent the last few decades adding layers of management and more committees. That machine is ill-suited to sudden change. Most universities will fail badly. They will look increasingly out-of-place.

— Daniel Lemire

In Portugal, we need to remove A3ES's role in approving individual degrees. A3ES should approve universities, schools, or departments and let them independently manage their offerings. Both the MIT and Coimbra reports support an iterative approach with continuous monitoring and feedback.

We urge MIT to revise its governance processes to promote more rapid curricular exploration, paired, of course, with thorough evaluation of the results. Departments need to be empowered to explore AI-aware substitutions and alterations to their curriculum on a regular basis without fixed, multi-committee, year-long review processes – or we will be left behind.

— The MIT Report on AI Use in Teaching, Learning, and Research Training

What I think we do not do enough in Portugal is seek industry feedback on the right skill set for software engineering. Traditionally, we expected to hear only about C# and .NET developers, which is why we—not me!—started ignoring industry. But based on job postings and conversations with industry leaders, I believe the following skills are needed today.

• Systems thinking and engineering.
• Foundations of programming, including code-quality metrics and experience with functional programming and specification languages, including dependent types.
• Foundations of distributed systems, including the web, APIs, and message queues.
• Foundations of databases, including real-world scalability, different database engines and models, and graph and semantic databases.
• Foundations of AI, including neural networks and linear algebra, LLMs, agent harnesses, evals, and classical machine-learning and reasoning techniques.
• Foundations of user experience, including how to design interfaces and services.
• Product-management skills, including experience running a project and defining quality metrics.
• Architectural design, so that graduates can critique the architecture and code produced by LLM-based agents and manage source-code complexity.
• Foundations of software security and safety, which are particularly important when vulnerability discovery is accessible to a layperson.

Of course, there are other topics to explore. But if a student masters these topics, I believe they will become an excellent professional. How to help students master them is a far more difficult question.

The illusion of understanding

A fundamental danger, as we’ve discussed, is that AI can allow students to bypass learning. Equally concerning is that students may internalize a transactional model in which assignments are outputs, teachers are evaluators, peers are optional, and knowledge (or an MIT degree) is an optimizable commodity to be acquired or produced as efficiently as possible.

— The MIT Report on AI Use in Teaching, Learning, and Research Training

Both the MIT and Coimbra reports conclude that students want clear, explicit rules about the kind of AI involvement expected in each learning activity. I disagree with that degree of micromanagement. Learning outcomes should state whether independent work is required, AI use is permitted, or effective AI use is required; assessment should, as far as possible, reflect those conditions. Outside assessment, students should be free to explore AI and reach their own conclusions. Instructors should communicate expectations, but should not dictate students' individual study practices.

But I do acknowledge the danger: students use completing an assignment or having the answer to a question as a proxy for success. The goal is not to obtain those particular answers, which could possibly be done by searching online, asking someone who took the course the previous year, or asking an LLM. The goal is to be able to solve a similar—or larger-scale—problem in the real world, where that particular question is only one small component. AI is fantastic for creating similar problems and giving feedback on a solution. It can be an excellent learning aid. So forbidding AI is not the solution; it is a shortcut that avoids what really matters: defining learning outcomes and robust methods for assessing whether those outcomes have been achieved.

When I taught functional programming, we had classes in which students came to the blackboard to share a solution so that we could discuss it. More often than not, students brought working solutions in their notebooks without knowing how they worked. Either they obtained them elsewhere or produced them through trial and error with the compiler. In either case, they had not achieved the intended learning outcome: they were supposed to be able to devise a readable, concise solution to any problem. Like machine-learning algorithms, students tend to overfit to the types of problems they encounter in class. Student representatives typically complain when exams contain types of questions they have not seen before. That is the opposite of what higher education should be. We are here to teach students how to handle the unknown using the foundations they have learned. The fact that many jobs over the last 50 years did not require a deep understanding of computer science cannot guarantee employment for the next generation.

All of us who teach at MIT will need to be prepared to help students understand both that the process of education is necessarily a productive struggle, and that the most important product of their education is not a GPA or a diploma but themselves: their personal growth and intellectual maturity and the development of their own imagination, insight, and judgment.

— The MIT Report on AI Use in Teaching, Learning, and Research Training

Organizing the curriculum

These competing demands on their time drive students to prioritize efficiency – and nothing could be more efficient than automating work through AI. But if students give in to that tempting option, they cheat themselves of the cognitive friction and productive struggle necessary for actual learning.

— The MIT Report on AI Use in Teaching, Learning, and Research Training

Curricula are typically structured in three stages: foundational courses, usually in the first year; scaling courses, usually in the second year; and applied courses, usually in the third and fourth years or at Master's level. Simas Kucinskas's Barbell approach for education with AI removes the middle of the degree.

One end of the barbell: courses that are deliberately non-AI. Work through proofs by hand. Read academic papers. Write essays without AI. It’s hard, but you build mental strength.

The other end of the barbell: embrace AI fully for applied projects. Attend vibecoding hackathons. Build apps with Cursor. Use Veo to create videos. Master these tools effectively.

— University education as we know it is over by Simas Kucinskas

The foundational courses that have existed for 40 years will still be relevant 40 years from now. We should keep them. According to Simas, the middle courses should disappear because they require too much effort to complete by hand, while completing them with AI does not lead to learning. While I agree with the latter point, I think whether they are worth the effort is debatable. Students need to spend time working by hand in order to understand what LLMs do on their own. Someone needs to, otherwise this is how they go rogue and destroy humanity.

In later, applied courses, students can use AI for all non-critical tasks. If they are implementing a Twitter clone, they should be able to use AI for all functional requirements and focus on the distributed-systems challenges. To this end, we should raise the bar for third-year courses, reaching the level of PhD research.

My instinct is that we need to raise the bar massively. A degree should conclude with work at the level of a 1995 PhD. That is doable in four years. Everything else should be short, doable in four months.

— Daniel Lemire

For this to happen, universities should invest more in giving students access to multiple LLM providers. The economic and environmental impact of doing so should be measured and communicated to students. This is also motivating: students get to build real products at university, see their ability to change the world, and build a portfolio for a market that increasingly looks only for senior developers.

Defining proper incentives

Most discussion of AI concerns student grading and assessment. That was the part of education we trusted and no longer do. I believe in starting from the foundations. For every course I teach this year, I will do the following:

First, I will rethink the skills that software engineers and computer scientists need within the relevant domain of expertise. For programming courses, I will write something like: "Students should be able to create, from scratch and without external assistance, any requested program in a familiar domain, using abstract data types, polymorphism, recursion, and higher-order functions." From this learning outcome, I will design an in-person method of assessment. For courses with 100 or more students, this might mean a written exam; for smaller courses, it might mean an in-person discussion. MIT recommends in-person discussions and states that departments should receive additional funding to support this costly form of assessment.

When designing an exam or in-person discussion, I will try to make the exercise as realistic as possible. For instance, instead of asking a student to write a recursive version of Fibonacci, as I did in the past, I can ask them to write a recursive function that determines which packages from a predefined list should be placed in a van with a weight limit to maximize a logistics company's profit. This is a real-world description of the knapsack problem. Students need to practise transferring foundational knowledge to concrete, real-world scenarios. Do not use the same exam format every year. Use a single problem in a distinct, unfamiliar real-world scenario each time. The more unfamiliar the assessment problem, the more prepared students need to be. Be clear about this from day one, so there are no surprises.

If I were designing an in-person discussion for a testing and validation course, I would discuss the challenges of testing AI chatbots and agents based on the foundations students had learned. This is a PhD-level challenge today, but students should be able to discuss it at this level after doing the work during the semester.

At the beginning of the semester, connections with industry professionals are important. They can help define a course's target skills, visit during the first two weeks to motivate students with real-world problems, and inspire exams and assignments based on those problems. A funny anecdote: in Software Construction, I always taught that Git commits should be granular and that students should not submit a project as a single commit. They ignored me. The following week, André Luis from GitLab told them exactly the same thing, and they adopted it. Industry participation is valuable even in the worst-case scenario, when professionals say nothing different—though that was certainly not the case with André.

How do you run classes throughout the semester to achieve the goals you have defined? MIT promotes continuous feedback, and Coimbra suggests active learning. I honestly do not have an answer. If students are motivated, many types of class can work. If they are not, active learning or continuous feedback alone will not help.

The recipe for motivation requires several ingredients. First, it requires everyone to be present in person. Those who cannot should register for online-only degrees, such as the Open University or, in Portugal, Universidade Aberta. The MIT report agrees:

Much of what our students gain from MIT is never spelled out in a syllabus or an assignment; it’s what they learn from living and working on our campus in each other’s company – the tacit expectations, habits, relationships, and values that inform how they learn to solve problems, exercise judgment, persevere through difficulty, and become members of an intellectual community. “Residential education” is powerful in part because it happens everywhere: in residence halls, living groups, sports teams, arts groups, clubs, and so on.

— The MIT Report on AI Use in Teaching, Learning, and Research Training

I have always been against mandatory attendance. However, I teach at a public university where students pay less than €1,000 in tuition, while my taxes cover the remaining €5,000 of the cost. We should fail students quickly and expel them sooner if they do not show the motivation and ability to learn, excluding health problems, parenthood, military service, and other protected circumstances. Right now, we can only bar them after three years of failure. In-person assessment functions as a form of mandatory attendance.

To boost morale and increase motivation, I would start by inviting someone from industry to talk about their daily work and the skills it requires. This should, of course, be adapted to each course. I would then cover the foundations, always connecting them to real-world applications. Even when teaching something as simple as if-then-else expressions, I can give a real-world example, such as home-automation workflow programming or checking whether a user is authenticated. In practical classes, students will use pen and paper. If it is up to me, they will throw that paper into the bin at the end of class, showing that getting the right answer to a particular problem is not the goal. The goal is for them to be able to solve the next problem by themselves. I will give students random and unusual problems; some might even be impossible. We have to exercise mental effort and critical thinking without offloading to AI and degrading our own skills.

Contrary to the MIT and Coimbra suggestions, I will not tell students how to use AI in their studies. That should be their decision. I will, however, specify in the learning outcomes whether independent work is required, AI use is permitted, or effective AI use is required. Ideally, students should be able to achieve all outcomes without AI, although perhaps not within the given time frame.

Recommendations for your course

1. Review learning outcomes every semester with industry professionals. Emphasize foundations early in the degree and real-world applications later. For each outcome, state whether independent work is required, AI use is permitted, or effective AI use is required.
2. Align grading with learning outcomes. Conduct assessment in person whenever practicable, ideally through oral and group settings that foster social interaction, while grading each student individually. Increase staffing and funding accordingly.
3. Make classes in person and interactive, with guest lecturers from industry or research labs from the first year onward.
4. Encourage students to explore agents and LLMs just as they explored IDEs and operating systems in the past. Curiosity is valuable, and students should decide how to use AI in their own studies.
5. Ensure that students build real-world projects with a positive social impact by the end of non-foundational courses.
6. Ensure that students work in teams that mirror real-world software-engineering projects.
7. Provide students with access to frontier models so they can prepare for the real world. Fund this transparently, and measure and communicate its economic and environmental impact.
8. Provide space for students to work in person and run their own clubs. Space is expensive at universities, but this is worth it.
9. Streamline curricular adjustments and reduce paperwork. Everyone, including instructors, is learning.

Paulo successfully defended his PhD thesis this Monday, wrapping up his dual PhD degree between Lisbon and Carnegie Mellon. Through out his PhD, Paulo studied the challenges that arise when writing modular robotics software using ROS. To help detect several types of misconfigurations, Paulo developed ROSpec, a specification language with Liquid Types that does not run. Instead, developers write the specification of the modules they create, and system integrators write the specification of their whole robot (the glue code). The type checker tells them whether it should work (w.r.t. the specification) or not.

Paulo also did a bunch of other stuff during his PhD, including an internship at Uber, but I believe ROSpec to be the highlight of his PhD. Good luck in your career, and at the next step, Sonar.

A equipa da Amália disponibilizou o seu modelo no HuggingFace, uma plataforma de partilha de modelos para se usar em casa, pelos mais aptos tecnològicamente.
O nosso primeiro-ministro indicou que esta versão ainda não responderia a perguntas, mas isto não é 100% verdade. Este modelo responde a perguntas, desde que cada um instale no seu computador. O estado neste momento não disponibiliza servidores para correr o modelo pelos portugueses.

Se for alguém mais familiarizado com linhas de comando, poderá simplesmente correr (graças ao Duarte Carmo) o seguinte comando:

llama cli -hf duarteocarmo/AMALIA-9B-0626-SFT-GGUF:Q4_K_M

Neste momento, pelo menos duas pessoas disponibilizaram um servidor que corre o modelo: temos a Amália do Duarte Carmo e a Amália do Henrique Macedo, ambas prontas a responder às vossas perguntas.

Mas mais uma vez, um político criticou o modelo por não estar a par das actualidades

Todos os modelos, sejam os Claudes ou GPTs, são treinados com dados até uma determinada data. Só conseguem responder com informação mais actualizada quando são treinados com a capacidade de recorrer a ferramentas externas (o famoso RAG).

Para dar esta funcionalidade ao Amália, eu — ou o Cursor, que programou esta funcionalidade por mim — criei um servidor intermédio, que recebe os pedidos do utilizador, e os envia à Amália, acrescentado alguns dados ao pedido: a data e hora actual, e a disponibilização de um serviço de procura.

Se a Amália decidir que precisa de algo, responde ao agente intermediário que precisa da informação X. O agente intermédio procura e volta a fazer o pedido à Amália, desta vez com o resultado da procura online. Assim que a Amália decidir que não precisa de mais procuras, a resposta é enviada ao utilizador final.

Este é o poder do RAG e, das minhas poucas experiências, parece que a Amália está bem preparada para ele.

If generative AI tools generate inappropriate language, plagiarized content, biased content, errors, mistakes, incorrect references, or misleading content, and that output is included in scientific works, it is the responsibility of the author(s). We have recently clarified our penalties for this. If a submission contains incontrovertible evidence that the authors did not check the results of LLM generation, this means we can’t trust anything in the paper. The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue. Examples of incontrovertible evidence: hallucinated references, meta-comments from the LLM (“here is a 200 word summary; would you like me to make any changes?”; “the data in this table is illustrative, fill it in with the real numbers from your experiments”)

— Thomas G. Dietterich

Just like Github, arXiv is where anyone can upload their scientific outputs. There’s a minimal verification to prevent Spam, but arXiv was never about gatekeeping content. Until now.

As I mentioned before, reputation is more important than ever in an age where text (and voice) is being produced cheaply, financed by monopoly-inducing LLM factories at a loss.

If the goal of arXiv is to provide an open alternative to the gatekeeping of journals, what is the open alternative to the gatekeeping of arXiv?

A U. Lisboa é a maior universidade portuguesa, mesmo na capital do país. É uma boa aproximação do que se passa no ensino superior português.

Como é que menos de metade dos alunos de uma licenciatura não a acabam no tempo previsto? Sei que não é a dificuldade que tem aumentado, pelo contrário: nunca foi tão fácil ter sucesso no ensino superior.

A minha hipótese é que os alunos não têm motivação. Estudam porque sim (sem verdadeiramente gostar dos tópicos), e dedicam o tempo e a atenção a tudo menos ao que importa. Estou a falar dos que estão fora dos 45%. Certamente há muitos alunos aplicados e que se esforçam. Mas devia ser quase a totalidade dos elementos.

Era interessante que fosse feito um estudo piloto em que os alunos eram obrigados a fazer um gap year a trabalhar. Penso que faria grande diferença na atitude dos alunos perante o desafio que é o ensino superior.

O governo fez aquilo que melhor sabe para resolver um problem — e eu concordo que é um problema em Portugal —, que foi legislar na nova proposta do Regime Júridico das Insitutições de Ensino Superior. Em particular,

Escolas médicas querem regime de excepção nas regras contra endogamia académica
“As unidades orgânicas que não tenham pelo menos 40% de docentes e investigadores de carreira licenciados ou doutorados noutra instituição de ensino superior ficam impedidas de contratar, independentemente do tipo de vínculo, nos três anos subsequentes à obtenção do grau de doutor, como docentes ou investigadores que nela tenham obtido todos os seus graus académicos.

Esta regra é um grande tiro no pé, prejudicando imensamente universidades no interior ou em cidades que só têm uma universidade. Imaginemos, alguém que termina o doutoramento na Universidade de Lisboa, pode ir para a Nova, para o ISCTE ou para outra das várias universidades que Lisboa tem. Já alguém que termine o doutoramento na UTAD tem obrigatòriamente de mudar de cidade. Imaginem o problema da UTAD com dificuldade em contratar professores, porque os doutorados que lá vivem não podem lá trabalhar.

Ora, existem vários motivos válidos para alguém não querer mudar de cidade (apoio à família, não querer que os filhos mudem de escola, two-body problem, ou porque acumulam com outra actividade profissional que é local. Esta medida prejudica gravemente quem está nestas situações.

O Conselho de Escolas Médicas Portuguesas concorda, pois os médicos não estão em regime de exclusividade e não querem mudar de serviço nem de cidade. E não são só os médicos!

E sim, eu sei que existe muita endogamia controlada (incluindo na Medicina), mas a solução não é bloquear contratações. Nem é delegar a contratação para elementos externos, como acontece com os painéis de peritos imparciais, mas que são escolhidos pela casa a dedo para valorizarem o que querem valorizar, numa táctica de tit-for-tat. Nós devemos dar às instituições a liberdade de definirem os seus objectivos e as estratégias e práticas que levem a esses objectivos. Mas devemos também avaliar e fiscalizar as decisões tomadas, com fortes implicações na carreira. Se a estratégia de contratação de uma entidade não funciona, é preciso entender o porquê e perceber se foi de má fé ou não.

Isso é o que devemos combater: a endogamia que foi feita por má vé, vs a endogamia que acontece por factor externos e justificáveis.

What may happen is that software development involves less coding than it has in the past because of AI. At least coding by humans. So BLS is probably right about a decline in the need for computer programmers. At the same time, if software developers spend less time doing actual coding they may have more time for higher level (if that is the right term) thinking and involvement in design. Unless AI starts doing more of that. So maybe we will not need more of them. Or perhaps AI will make it possible for more people to be software developers who wouldn’t be that now. We’ll see I guess.

— Computer Programming or Software Development by Alfred Thompson

Alfred analyses the difference between a programmer and a software developer. AI is replacing programmers (those that implement features identified by software developers), but not Software Engineers.

On the other hand, we might not be preparing our SE students for the next decade. We have good, core CS and Programming courses. But advanced courses are not up to par with what the market needs. This aligns with the Barbell approach, which is the closest I have seen to a good path for our SE education. We need good, pen-and-paper, fundamental courses, and we need up-to-date advanced courses that make use of AI and whatever comes next.

The main problem is that technology is moving faster than Universities can adapt. Most professors are researchers in their own niche, and most are not doing Software Engineering, but they do teach it. We need more cutting-edge engineers to come back to universities to teach.

Here in Portugal, we have incentives not to hire professionals (I am fighting this locally, and got two real-world engineers to teach Functional Programming with me) and our degrees have to stay static for three to four years. This does not work for this day and age when the development process changes so frequently, and professors are so busy to actually get some hands on experience. I am also fighting that, but that’s for some other post.

How do you acquire the fundamental computer skills to hack on a complex systems project like OCaml? What’s missing and how do you go about bridging the gap?

— KC Sivaramakrishnan

KC gives several resources for students to get up to speed with contributing to OCaml.

One of the interesting resources is MIT’s the Missing Semester. This semester I created our own version of this course, covering git, docker, VMs, terminal/bash, testing, static analysis and LLMs for code.

While we cover how to do a Pull Request, I don’t believe students are ready to actually contribute. Reading large codebases is a skill that even our graduate MSc students don’t have. Courses are designed to be contained, with projects that need to be graded with few human effort, resulting in standard assignments for all the students.

I would love to run something like the Fix a real-world bug course Nuno Lopes runs. But being able to review so many PRs is a bottleneck in a regular course.

To really understand a concept, you have to “invent” it yourself in some capacity. Understanding doesn’t come from passive content consumption. It is always self-built. It is an active, high-agency, self-directed process of creating and debugging your own mental models.

— François Chollet (via Simon Willison)

It’s a rephrasing of our “The best way to understand something is to teach it to someone else”. And that’s why I still love my job.

If ChatGPT can produce research papers that are indistinguishable from what most scientists can write, then maybe scientists can focus on actually advancing science—something that ChatGPT has thus far proven unable to do.

— Beyond papers: rethinking science in the era of artificial intelligence by Daniel Lemire

Looking at the proceedings of our conferences over the past few years, I find that most of the papers are simply uninteresting. Moreover, it seems that every first-year PhD student is now required to write a systematic review on their topic — supposedly to learn about the field while producing a publication.

Let me be blunt: every systematic review I’ve read has felt like a waste of time. I want to read opinionated reviews written by experts — people who have seen enough to have perspective — not by PhD students who have just skimmed the past decade of papers on Google Scholar.

We need far fewer papers (I’m doing my best to contribute to that cause), and the ones we do publish should be bold, revolutionary, and even a little irreverent. We need innovation and the courage to break expectations. Incremental research has its place, but that doesn’t mean it always needs to be published.

To make this possible, evaluation committees — both nationally and within universities — must rethink their processes to move away from bean-counting metrics. Our current incentive system discourages genuine peer review, and even when proper reviews happen, they often waste effort on work that adds little value.

Otherwise, yes — the bean-counting-reinforcement-learning AIs will take our jobs.

No Politécnico de Coimbra subiram a taxa de matrícula de 30 para 125 euros. Alunos de Mestrado e Doutoramento pagam até 500 euros de taxa de entrega de tese.

No caso das licenciaturas, estas taxas servem para as universidades públicas receberem mais dinheiro do que a propinas que está definida por lei. A nível de doutoramento, serve para manter o valor da propina naquele que a FCT suporta nas suas bolsas (2750 euros).

A verdade é que os alunos vêem um preço anunciado, e depois é-lhes impossível acabarem o curso pagando apenas esse valor. É literalmente publicidade enganosa.

Precisamos de duas mudanças: eliminação das taxas por parte das Universidades e Politécnicos, englobando esse custo na propina. Um aluno pagando a propina, deve conseguir ter acesso a assistir às aulas, ser avaliado e obter o diploma, sem qualquer taxa.

E o estado precisa de majorar o financiamento das universidades, que claramente têm de recorrer a estas acções eticamente discutíveis para manter a sustentabilidade económica que lhes é exigida pelo Tribunal de Contas.

— AI is a Floor Raiser, not a Ceiling Raiser

Every day I am convinced that Software Engineering should be taught without AI. AI can give you answers to easy problems. But you won’t be able to create mental models of how things work, which will help you solve hard problems.

This next semester I am teaching a course on practices and tools in Software Engineering (my take will be inspired by MIT’s The Missing Semester). AI usage will be one of the topics, where we will explore MCP, IDE integrations and AI-assisted documentation.

But I have no idea how to write assignments for other topics. It is very likely that an AI will be able to complete the assignment without any human intervention. If students opt to do that (they will, that’s the faith I have in our grade-oriented system), they will not achieve the learning outcomes.

Students will question: if AI can do these tasks, why should we learn it? Well, math teachers in school still assigned me problems that machines could already solve by then. But creating mental models of how things work is essential in education.

Now the real question is about the incentives. We should assess whether students can use their mental models, and not whether they can solve the task. Especially with 100 students, where exams or take-home assignments are the norm.

In 2010 Carmen Reinhart and Kenneth Rogoff published Growth in a Time of Debt. It’s arguably one of the most influential economics papers of the decade, convincing the IMF to push austerity measures in the European debt crisis. It was a very, very big deal.
In 2013 they shared their code with another team, who quickly found a bug. Once corrected, the results disappeared.
Greece took on austerity because of a software bug. That’s pretty fucked up.

— How do we trust our science code? by Hillel Wayne

As more and more scientific publications are dependent on code, trusting code is more and more needed. Hillel asks for solutions, I propose to tackle the problem in two fronts.

1 – More engineering resources

Writing production-level quality software requires larger resources (usually engineerings, but also some tooling). Most scientific software is written once and read never. Some PhD or MSc student writes a prototype, shows the plots to their advisors who write (some or most of) the paper. It’s rare for senior researchers to inspect other people’s code. In fact, I doubt any of them (except if they teach software engineering principles) has had any training in code inspection.

We need research labs to hire (and maintain) scientific software engineering teams. For that to happen, funding has to be more stable. We cannot rely on project funding that may or may not be awarded. We need stable funding for institutions so they can maintain this team and resources.

2 – More reproducibility

Artifact Evaluation Committees are a good addition to computer science conferences. Mostly comprised of students (who have the energy to debug!), they run the artifacts and verify whether the results of the run justify the results presented in the paper. Having done that myself in the past, it is very tricky to find bugs in that process. Mostly we verify whether it will run outside of your machine, but not whether it is rightly implemented.

What would help is to fund reproduction of science. Set 50% of the agency funding for reproducibility. Labs that get these projects should spend less than the original project to reproduce the results (and most of the challenging decisions are already made). In this approach, we will have less new research, but more robust one.

Given how most of the CS papers are garbage (including mine), I welcome this change. We need more in-depth strong papers that move the needle, and less bullshit papers that are just published for the brownie points.

Overall we need better scientific policies with the right incentives for trustworthy science. I wonder who will take this challenge on…

Recommended audience: CS students

Austin Henley shares some properties of a good side project. Personally, I think having a clear shippable objective is what most people lack, and prevents them from ever being complete.

I remember having side-projects suggestions during my courses. Maybe that’s something I have to incorporate in mine.

Most of what I’ve learned during my degree was doing side-projects. From competing in hackathons, creating a junior company, organizing conferences, doing a couple of research internships, and doing some freelancing work, these projects all taught me something that was not in the syllabus. That’s what separates you from the average student, and what will get you a good job in a world where unemployed software engineers are aplenty.

Joshua Barretto shares a really interesting list of possible side projects:

• Regex engine (5 days)
• x86 kernel (2 months)
• Gameboy emulator (3 weeks)
• Gameboy advance game (2 weeks)
• Chess engine (5 days)
• Physics engine (1 week)
• Voxel engine (2 weeks)
• GUI Toolkit (3 weeks)
• Posix shell (5 days)
• Dynamic interpreter (2 weeks)
• Compiler (3 months)
• Threaded Virtual machine (1 week)
• Text editor (4 weeks)

The last four will give you an heads up in the programming language world. I might even have an internship for you.

Perhaps you’re a user of LLMs. But I might suggest resisting the temptation to use them for projects like this. Knowledge is not supposed to be fed to you on a plate. If you want that sort of learning, read a book – the joy in building toy projects like this comes from an exploration of the unknown, without polluting one’s mind with an existing solution.

Recommended audience: Startups and large companies who intend to sell software to universities.

Most SAAS is sold on a per-seat basis. But this does not scale to universities, as we have a large number of possible seats, but most of them (students, possibly from different scientific areas) do not use the software, at least for it to be worthwhile.

On the other hand, unpredictable costs (when paying per activity) is also something that does not work, as we need other budget it yearly.

Chris Siebenmann has a really good write up on this issue, which I recommend if you manage or sell to universities.

My first day of uni, I received these two t-shirts designed by the student group.

While I completely forgot about the top one, I keep the bottom one near my heart. While I found it amusing, I did not find it to be true. I did learn a lot during these years, and I gained much more than the degree (which is only required for the Portuguese bureaucratic system where Simon Peyton Jones couldn’t even get a position as Assistant Professor).

Now 19 years later, I no longer find it to be funny. I feel the scholarly spirit is dying and young people do not care about learning or knowledge. They care only about grades and getting the degree. And GPT is the TLA that takes them from the donkey without the diploma to the donkey with the diploma.

This tradition started back when I was a student. I installed random software for each of the 5 courses I took every semester. I ended up with wasted disk space, random OS configurations and always a complete mess in my $PATH.

So I started formatting my Macs at the end of every semester. And I continue doing that today. Being a professor, I also deal with the software baggage every same semester — otherwise I would probably format it every year.

Most of the people I know think this is insane! Because they spend days in this chore, they avoid it as much as possible, often delaying it so much that they end up buying a new computer before considering formatting. And they also delay buying a new computer for the same reason.

My trick is simple: I automate the process as much as possible, such that it takes ~20 minutes now to format and install everything, and another 2 hours to copy all data and login into the necessary accounts. And you can watch a TV Show while doing it.

I keep a repository with all my dot file configurations, which also contains scripts to soft link all my configurations (usually located at $HOME/Code/Support/applebin) to their expected location ($HOME). This process also includes a .bash_local or .zsh_local where I introduce machine or instance-specific details that I don’t mind losing when I format it in 6 months. Long-lasting configurations go in the repo.

If the machine runs macOS, I also run a script that sets a bunch of defaults (dock preferences, Safari options, you name it) that avoid me going through all settings windows and configuring it the way I like it.

But the most useful file is my Brewfile, that contains all the apps and command-line utilities I use. I should write another usesthis, where I go through all the apps I have installed, and why.

My process starts with copying my home directory to an external hard-drive (for restoring speeds). During this process I usually clean up my Downloads and Desktop folders, which act as more durable /tmp folders. When it’s done, I reset my MacBook to a new state. I then install homebrew and Xcode command line utilities (for git), I clone my repo and run the setup script. At the same time, I start copying back all my documents from the external drive back to my Mac. Then it’s time to do something else.

Two hours later, I can open the newly installed apps and login or enter registration keys, and make sure everything is working fine.

Now I’m ready for the next semester!

“scientists often use code generating models as an information retrieval tool for navigating unfamiliar programming languages and libraries.” Again, they are busy professionals who are trying to get their job done, not trying to learn a programming language.

— How scientists learn computing and use LLMs to program: Computing education for scientists and for democracy

Very interesting read, especially since we teach programming to non-CS students, which is fundamentally different. Scientists are often multilingual (Python, R, bash) and use LLMs to get the job done. Their goal is not to write maintainable large software, but rather scripts that achieve a goal.

Now I wonder how confident they are that their programs do what they are supposed to do. In my own research, I’ve found invisible bugs (in bash, setting parameters, usually in parts of the code that are not algorithmic) that produce the wrong result. How much of the results in published articles is wrong because of these bugs?

We might need to improve the quality of code that is written by non-scientists.

Advice people with long careers on what worked for them when they were getting started is unlikely to be advice that works today. The tech industry of 15 or 20 years ago was, again, dramatically different from tech today. I used to joke that if you knew which was was up on a keyboard, you could get a job in tech. That joke makes no sense today: breaking into the field is now very difficult, and getting harder every year.

— Beware tech career advice from old heads, by Jacob Kaplan-Moss

The industry is undervaluing junior developers, by thinking LLMs can do their work. This is true at this instant, but junior developers have the potential to become senior developers.

I still remember years when my team did not have interns at Uber; and years when we did. During the time we did: energy levels were up, and excluding the intern I’d wager we actually did more. Or the same. But it was a lot more fun. All our interns later returned as fulltime devs. All of them are now sr or above engineers – at the same company still (staying longer than the average tenure)

— Gergely Orosz

It is up to your faith whether LLMs can eventually be promoted to senior developers (or management). And if you believe it, you may need to reconsider your own job.

[…] the top learning objective was for their students to understand that websites can be built from databases.

I’m pretty sure that the most popular programming language (in terms of number of people using it) on most campuses is R. All of Statistics is taught in R.

End-user programmers most often use systems where they do not write loops. Instead, they use vector-based operations — doing something to a whole dataset at once. […] Yet, we teach FOR and WHILE loops in every CS1, and rarely (ever?) teach vector operations first.

— CS doesn’t have a monopoly on computing education: Programming is for everyone by Mark Guzdial

The main take away is that you do not teach Programming 101 to non-Software Engineering/Computer Science the same way you teach to those students. The learning outcomes are different, and so should the content.

Funny how Functional Programming (via vectorized operations) is suggested to appear first than imperative constructs like for or while. This even aligns with GPU-powered parallelism that is needed when processing large datasets.

Food for thought.