The Neural Information Processing Systems Foundation's annual conference, NeurIPS, witnessed an unprecedented surge in demand for its 2018 edition. The rapid sell-out of tickets underscored the burgeoning excitement and rapid advancements within the field of machine learning and artificial intelligence. Amazon, a prominent player in this domain, had a significant presence at the conference, with its researchers sharing valuable insights into the state of AI.

The Growing Momentum of AI

Bernhard Schölkopf, chief machine learning scientist for Amazon's retail division and a member of the NeurIPS advisory board, commented on the transformative shift in AI research. He noted that "AI research was somewhat stuck until people noticed that by using huge datasets, serious computing infrastructure, and state-of-the-art machine learning methods, very real progress can be made." This realization has propelled NeurIPS from being a premier machine learning conference to arguably the most sought-after AI conference.

Valuable Engagement at NeurIPS

Alex Smola, machine learning director for Amazon Web Services' Deep Engine group and an area chair at NeurIPS, highlighted the unique value proposition of the conference. He stated, "The poster sessions are probably the best part of NeurIPS, due to the higher degree of engagement you can have with the authors and discuss technical details." Smola also emphasized the importance of workshops, which have evolved into "mini-conferences" attracting hundreds of attendees.

Advancements in Conversational AI

Dilek Hakkani-Tür, a senior principal scientist in the Alexa AI group, observed a significant increase in submissions related to conversational AI. She noted, "Paper submissions are more than 50 percent higher this year." Hakkani-Tür highlighted the shift towards more "chit-chat-based or social-bot types of systems," while acknowledging the enduring difficulty in creating machines capable of open-domain conversation. Amazon's active participation in organizing and programming the Conversational AI workshop further demonstrated its commitment to this rapidly evolving area.

Diversification of Machine Learning Approaches

Ralf Herbrich, director of machine learning science for Amazon's Core AI group and the demonstrations and competitions chair at NeurIPS, observed a healthy trend towards diversification in machine learning approaches. He noted a shift away from the near-exclusive focus on deep learning seen in previous years. Two particular areas that garnered increased attention were Bayesian learning and spiking neural networks.

Bayesian Learning: Efficiency and Reasoning

Herbrich explained the significance of Bayesian learning, which not only discerns statistical patterns but also estimates the probability of their accuracy. This allows Bayesian systems to learn more efficiently, especially in scenarios with sparse data, such as Amazon's extensive product catalog. "What you need there is something that is closer to human reasoning, which can learn from very few examples," Herbrich stated, underscoring its relevance for Amazon's customer-centric approach of offering "more selection."

Spiking Neural Networks: Power Efficiency for the Edge

The growing interest in spiking neural networks was also a key observation. Unlike conventional deep neural networks, spiking neural networks simulate the human brain more closely, where neurons fire only when their contribution is relevant. This makes them significantly more power-efficient. Herbrich illustrated this with the example of a device like the Echo, stating, "As more and more compute moves to the edge, energy efficiency becomes an important criterion because it constrains how often and where you can use it."

Robustness and Societal Implications

Inderjit Dhillon, an Amazon fellow in the Search Technologies group and a professor of computer science, served as a senior area chair for NeurIPS. He noted a rise in research on adversarial methods, aimed at making neural networks more robust against manipulation, particularly in safety-critical applications like self-driving cars. Dhillon also observed a growing discussion around the societal implications of AI, including fairness and explainability, though he noted these were more prominent in invited talks and workshops than in technical papers. He expressed hope that more researchers would address these critical societal issues with a technical perspective.

Amazon's Research Contributions

Amazon researchers actively contributed to the NeurIPS program, with co-authored papers and participation in various committees and boards. This involvement provided them with a unique vantage point to assess the overall trends and directions of machine learning research, reinforcing Amazon's position at the forefront of AI innovation.

Key Takeaways from NeurIPS 2018

The NeurIPS 2018 conference highlighted several key trends: a continued prevalence of deep learning, albeit with a growing diversification of approaches; a significant focus on fairness, diversity, and AI for social good; the sustained uptrend in reinforcement learning; and a renewed emphasis on error bounds and confidence intervals, signaling a return to Bayesian machine learning principles. The workshops covered a wide array of application domains, from medical imaging and social good to mobile devices and computer architecture, showcasing the broad impact of machine learning across industries.

In summary, NeurIPS 2018 demonstrated the dynamic and rapidly evolving nature of artificial intelligence, with Amazon playing a pivotal role in shaping its future through its research contributions and active participation in the global AI community.

Tag: neurips

NeurIPS 2018: A Glimpse into the Evolving Landscape of Artificial Intelligence and Amazon's Role