Image by pch.vector on Freepik
Machine learning is a large field with new research coming out frequently. It’s a hot field where academia and industry continue to experiment with new things to improve our everyday lives.
In recent years, generative AI has been changing the world through the application of machine learning. For example: ChatGPT and Stable Diffusion. Even if 2023 is dominated by generative AI, we should be aware of more advances in machine learning.
Here are the best machine learning papers to read in 2023 so you don’t miss the upcoming trends.
1) Learning the beauty in songs. Nervous singing voice beautifier
Singing Voice Beautifying (SVB) is a new task in generative AI that aims to enhance an amateur singing voice into a beautiful one. That is exactly the research goal of Liu et al. (2022) when they proposed a new generative model called Neural Singing Voice Beautifier (NSVB).
NSVB is a semi-supervised learning model that uses a hidden mapping algorithm that acts as a pitch corrector and tone enhancement. The work promises to improve the music industry and is worth checking out.
2) Symbolic disclosure of optimization algorithms
Deep neural network models have become bigger than ever, and a lot of research has been done to simplify the learning process. Recent research by the Google team (Chen et al. (2023)) proposed a new neural network optimization called Lion (EvoLved Sign Momentum). The method shows that the algorithm is more memory efficient and requires a smaller learning rate than Adam. It’s a great piece of research that shows a lot of promise that you shouldn’t miss.
3) TimesNet. temporal 2D-variational modeling for general time series analysis
Time series analysis is a common practice in many businesses. For example, price forecasting, anomaly detection, etc. However, there are many challenges in analyzing temporal data based only on current data (1D data). For this reason, Wu et al. (2023) propose a new method called TimesNet to transform 1D data into 2D data, which achieves great efficiency in experiment. You should read the paper to better understand this new method, as it will help in many future time series analyses.
4) OPT. Open pre-trained transform language models
We are currently in the generative AI era, where many large language models have been intensively developed by companies. Mostly such research will not release their model or will only be commercially available. However, the Meta AI research group (Zhang et al. (2022)) is trying to do the opposite by publicly releasing an Open Pre-trained Transformers (OPT) model that can be compared to GPT-3. The paper is a great start to understanding the OPT model and research details as the group records all the details in the paper.
5) REALTabFormer. Creating realistic relationships and tabular data using transformers
The generative model is not limited to the creation of text or images, but also to the creation of tabular data. This generated data is often referred to as synthetic data. Many models have been developed for creating synthetic tabular data, but almost no models for creating synthetic relational tabular data. This is exactly the purpose of the research of Solatorio and Dupriez (2023). creating a model called REaLTabFormer for synthetic relational data. Experiments have shown that the result closely approximates an existing synthetic model that can be extended to many applications.
6) Is reinforcement learning (not) for natural language processing? benchmarks, baselines, and building blocks for natural language policy optimization.
Reinforcement learning is conceptually a great choice for a natural language processing task, but is it true? This is the question that Ramamurthy et al. (2022) try to answer. The researcher presents various libraries and algorithms that show where reinforcement learning techniques have an advantage over the supervised method of NLP tasks. It’s a recommended read if you want an alternative for your skills.
7) Tune-A-Video. single-frame tuning of image diffusion models for text-to-video generation
Text-to-image was big in 2022, and 2023 was predicted to be text-to-video (T2V). Wu et al. (2022) shows how to extend T2V to multiple approaches. The research proposes a new Tune-a-Video method that supports T2V tasks such as subject and object switching, style transfer, attribute editing, etc. It’s a great paper to read if you’re interested in text-to-video research.
8) PyGlove. Effective sharing of ML ideas as code
Effective collaboration is key to the success of any team, especially in the ever-growing complexity of machine learning. To develop efficiency, Peng et al. (2023) present a PyGlove library to easily share ML ideas. The PyGlove concept encapsulates the ML exploration process through a list of patching rules. The list can then be used in any rehearsal scene, improving team efficiency. It’s research that tries to solve a machine learning problem that most people haven’t yet, so it’s worth a read.
8) How close is ChatGPT to human experts? Comparison Corpus, Evaluation and Discovery
ChatGPT has changed the world so much. It’s safe to say that the trend will go up from here, as the public is already in favor of using ChatGPT. However, how does ChatGPT currently perform compared to human experts? That is exactly the question that Guo et al. (2023) try to answer. The team tried to collect data from experts and quick results from ChatGPT, which they compared. The result shows that there are implicit differences between ChatGPT and experts. The research is something that I think will hold in the future as the generative AI model continues to grow over time, so it’s worth a read.
2023 is a great year for machine learning research given the current trend, especially generative AI such as ChatGPT and Stable Diffusion. There is a lot of promising research that I think we should not miss because it has shown promising results that could change the current standard. In this article, I have shown you the 9 best ML papers to read, ranging from generative model, time series model to workflow efficiency. I hope that helps.
Cornelius Judah Vijaya is an assistant data science manager and data writer. Working full-time at Allianz Indonesia, he enjoys sharing Python and Data tips through social media and written media.