Programming Languages: Design and Implementation is probably the top programming language conference (maybe tied with POPL, but POPL is more theoretical). This year I was invited to be part of the Program Committee that would review papers, probably due to my work last year in the Artefact Evaluation Committee. So I would recommend senior PhD Students and junior faculty to contribute service in the same manner as evaluating artefacts is very useful for sustainable science.

Impostor Syndrome

And of course Impostor Syndrome kicks in. I am reviewing papers from people who have multiple PLDI publications and many years in these topics. How can I criticise their work? And no, I am not vingativa of all my PLDI rejections over the years. I really want good papers to be published, as it’s how the field progresses, and we have even more interesting ideas to explore. On the other hand, the idea of peer reviewing is to minimize the flaws found in published papers. And reviewing is also a best-effort. Ideally, we would replicate the experiments. And I don’t mean re-running the scripts to generate results and tables and figures, like what the AEC does, but rather re-implement everything from scratch to validate the results. However, there is no time, and no funding for that task. Maybe funding agencies could allocate half of the budget for replication of results (doing less broad research, and being more confident in the fewer results that we get), but not all areas require such validation efforts. So it’s almost guaranteed that my reviews will be flawed, but I gave my best.

Workload

Because I did not submit any paper, I was assigned all papers authored by PC members (PC members are held to an higher standard in terms of number of reviewers AFAIK). That was a total of 10 papers, which was possible to review in the allotted time. Reviewing occurred during our Winter Exam Phase (Jan-Feb), which is the perfect time for us in Portugal. Unfortunately, the next phases (reviewing rebuttals, and making the final decisions) occurred during the most busy weeks of the Spring semester which, together with a small health issue, made me miss deadlines. Ideally, we would have backup reviewers, but that would increase the time-to-response. Fortunately, there are Area Chairs that chime in and can ask for extra reviews.

Of the 10 papers, I felt capable of being a proper reviewer for almost all of them. They would either be about my PhD topic, or things I am working on.