participatory-budgeting/review_2/review_2.tex

\documentclass[11pt,a4paper]{article}

\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{csquotes}
\usepackage[margin=1in]{geometry}
\usepackage{hyperref}
\usepackage{ulem}

\begin{document}

\section{Summary}
\label{sec:Summary}

The paper starts with providing the motivation for studying the traveling
tournament problem (TTP) and giving an example for its application to a real
world scenario. The complexity of the problem is introduced and opens the stage
for algorithms that solve the traveling tournament problem in a reasonable
amount of time. Next, the problem is defined by formulating the inputs,
constraints that have to be satisfied by solutions and the goal. An example
illustrating the problem and a solution is given. Simulated annealing is
discussed as the first approach to solving TTP. The search for solutions is
allowed to `backtrack' in a sense so that escaping local minima is possible. The
size and frequency of such moves is decreasing over time.  Additional techniques
for refining the search for solutions are discussed and referenced in
pseudocode. A parallel variation of the simulated annealing algorithm is
analyzed in section 4, giving better performance overall. The third approach ant
colony optimization has multiple iterative algorithms searching for a solution
randomly. Shorter paths are traveled more often than longer paths and are
therefore more promising. Due to positive feedback, more and more instances are
prioritizing the shortest path found so far. Section 6 compares the performance
of the three different approaches to solving TTP and the conclusion gives a
short outlook on promising research directions for the future.

\section{Evaluation}
\label{sec:Evaluation}

The paper is very well written, giving a carefully selected introduction to TTP
and meta-heuristic methods. Due to the structure being clear and
straightforward, following the descriptions and discussions is effortless.

The abstract has all the necessary information, hinting at the complexity of
TTP, possible solutions and the structure of the paper.

The introduction touches upon the motivation for solving TTP. I would suggest
including a short sentence or two about the structure of the paper (like in the
abstract) and a transition to the next section, which would aid the reader in
keeping context. I would also suggest including a short description of what
meta-heuristics are because the term is used later on but never explained.

Section 2 establishes an intuitive problem setting without going into too much
detail. The sentence beginning with \enquote{Now the goal \dots} could be
formulated differently because it sounds like teams have to play away games
exclusively between leaving and returning home. If possible, giving a short
sentence to transition to section 3 would add more `natural flow' to the paper.

The section on simulated annealing is very informative and written in a way that
is easy to understand for a layman. Especially the two subsections (3.1 and 3.2)
help with gaining a deeper understanding. When walking through the pseudocode in
figure 2, a fifth point on what happens after point four would be beneficial.
For example a sentence about how long the process is looped and when it
terminates might be a possibility. For section 3.2 I would avoid having two
headlines without any text between them. A sentence about what these advanced
techniques accomplish should suffice. In section 3.2.1 it is mentioned that the
parameter $w$ is chosen but not how it is chosen (arbitrarily or according to a
rule?). The same applies to section 3.2.3 where it is not entirely clear how the
number of maximum reheats (\emph{maxR}) is set.

The figure in section 4 explains the concept of PBSA very well.

Section 5 has a well written introduction which allows the reader to gain an
intuitive understanding of this approach. It might not be clear to the reader,
however, what the pheromone matrix contains (solutions and their pheromone
levels?). For section 5.2 I would suggest mentioning whether unsafe backjumping
has disadvantages regarding the quality of solutions.

Section 6 is short and to the point and succinctly compares the performance of
the three approaches. If a performance evaluation of ACO on the same dataset as
in figure 4 could be found, the comparison would be even more meaningful.

For the conclusion it is generally best to avoid introducing new references and
terms. I would therefore suggest expanding on future research directions like
the last sentence does. The sentence on recent advances might belong more to the
introduction than the conclusion.

The language used throughout the paper is consistently on a high level, although
small typing errors can be found (e.g., section 1 \enquote{\dots with the
constraint satisfaction \sout{what} that makes\dots}; section 3.2.2
\sout{devides} divides).

The references are used correctly and the bibliography provides the necessary
information. The paper is within the 7-10 page bound. The figures are a bit
blurry, so either trying to include a higher resolution version or rewriting
some of the figures in \LaTeX\ (e.g., the table in fig. 1 and the algorithm in
fig. 2) can be an improvement. The figures are still readable so this is just an
aesthetic suggestion.

Overall the paper is of very high quality and almost no points of criticism can
be found. The suggestions above are minor issues only.

\section{Minor issues}
\label{sec:Minor issues}

\begin{itemize}\setlength{\itemsep}{0pt}
    \item Some spelling mistakes (see section~\ref{sec:Evaluation} for
        suggestions)
    \item Blurry images
    \item Conclusion introducing new terms
    \item Minor additions to expand on definitions (the weight parameter $w$ and
        the pheromone matrix)
\end{itemize}

\end{document}