The author reflects on their machine learning class and shares several open research problems that emerged during the semester. They challenge the traditional reliance on data-generating distributions, advocating for a design-based approach to machine learning. This perspective shifts the focus from natural randomness to intentional randomness created by engineers. The author suggests that this approach can lead to better decision-making frameworks, such as an improved version of Neyman-Pearson decision theory, and highlights the inadequacy of current studies in terms of power when evaluating population-level outcomes. The piece also addresses gaps in machine learning theory, particularly regarding the persistent lack of consensus on phenomena observed in competitive testing. The author points out that while robust trends appear in benchmarks, existing theories fail to explain them. They draw an analogy to phenomenologists in physics, who develop models to explain trends, suggesting that machine learning could benefit from similar speculative approaches. Another theme is the limitation of average-case evaluations, which the author believes trap researchers in statistical methods and hinder innovative assessment strategies. In exploring LLMs (large language models), the author identifies a significant opportunity in optimizing reasoning processes. Current methods primarily use policy gradients to improve performance on tests, but the author argues this approach wastes efficiency and overlooks better optimization strategies. They invite researchers interested in this issue, emphasizing the potential for more effective methods. Lastly, the author stresses the importance of developing open-source, open-corpus language models, positioning this as a critical challenge in applied machine learning. They believe that with the right focus, it's possible to create high-performing models with fewer resources, which could alter the current computational landscape and geopolitical implications of AI development.