Real-Time Machine Learning: How Streaming Platforms Power AI Models
Keywords:
data streaming, fraud detection, IoT dataAbstract
Real-time machine learning has revolutionized how organizations extract value from their data by enabling faster and more responsive decision-making. Traditional batch-processing models, which handle large sets of data in discrete intervals, struggle to keep up with dynamic environments where data is constantly changing. In contrast, real-time machine learning continuously processes data as it streams in, allowing AI models to adapt & learn from new information in near real-time. This capability has become crucial for industries like e-commerce, finance, healthcare, and logistics, where rapid decision-making can have a significant impact on operations and customer experience. Streaming platforms such as Apache Kafka, Apache Flink, and Amazon Kinesis are central to this shift, providing the infrastructure necessary for real-time data ingestion, complex event processing, and predictive model scaling. These platforms allow data engineers and scientists to handle high-velocity data streams with minimal latency, making them indispensable for processing vast amounts of data efficiently. By enabling real-time data processing, these platforms help bridge the gap between raw data & actionable insights, offering scalability and fault tolerance that ensures the reliability of the system. Real-time machine learning empowers organizations to make timely, data-driven decisions, such as detecting fraud in financial transactions, personalizing customer experiences in e-commerce, or monitoring patient health in healthcare settings. The ability to continuously process data means that models can evolve and improve as new information is received, ensuring they remain relevant and accurate. This constant adaptation provides organizations with a strategic advantage, helping them stay competitive in fast-moving markets. Ultimately, real-time machine learning powered by streaming platforms enhances decision-making, streamlines operations, and unlocks the full potential of data, allowing businesses to deliver timely insights and make smarter, faster decisions across various sectors.
Downloads
References
Tien, J. M. (2017). Internet of things, real-time decision making, and artificial intelligence. Annals of Data Science, 4, 149-178.
Mohammadi, M., Al-Fuqaha, A., Sorour, S., & Guizani, M. (2018). Deep learning for IoT big data and streaming analytics: A survey. IEEE Communications Surveys & Tutorials, 20(4), 2923-2960.
Singh, J. (2014, March). Real time BIG data analytic: Security concern and challenges with Machine Learning algorithm. In 2014 Conference on IT in Business, Industry and Government (CSIBIG) (pp. 1-4). IEEE.
Nair, L. R., Shetty, S. D., & Shetty, S. D. (2018). Applying spark based machine learning model on streaming big data for health status prediction. Computers & Electrical Engineering, 65, 393-399.
Fowers, J., Ovtcharov, K., Papamichael, M., Massengill, T., Liu, M., Lo, D., ... & Burger, D. (2018, June). A configurable cloud-scale DNN processor for real-time AI. In 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA) (pp. 1-14). IEEE.
Degris, T., Pilarski, P. M., & Sutton, R. S. (2012, June). Model-free reinforcement learning with continuous action in practice. In 2012 American control conference (ACC) (pp. 2177-2182). IEEE.
Vaughan, A., & Bohac, S. V. (2015). Real-time, adaptive machine learning for non-stationary, near chaotic gasoline engine combustion time series. Neural Networks, 70, 18-26.
Kroll, B., Schaffranek, D., Schriegel, S., & Niggemann, O. (2014, September). System modeling based on machine learning for anomaly detection and predictive maintenance in industrial plants. In Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA) (pp. 1-7). IEEE.
Cai, H., Ren, K., Zhang, W., Malialis, K., Wang, J., Yu, Y., & Guo, D. (2017, February). Real-time bidding by reinforcement learning in display advertising. In Proceedings of the tenth ACM international conference on web search and data mining (pp. 661-670).
Hazelwood, K., Bird, S., Brooks, D., Chintala, S., Diril, U., Dzhulgakov, D., ... & Wang, X. (2018, February). Applied machine learning at facebook: A datacenter infrastructure perspective. In 2018 IEEE international symposium on high performance computer architecture (HPCA) (pp. 620-629). IEEE.
Kelly, J., Gooding, P., Pratt, D., Ainsworth, J., Welford, M., & Tarrier, N. (2012). Intelligent real-time therapy: Harnessing the power of machine learning to optimise the delivery of momentary cognitive–behavioural interventions. Journal of Mental Health, 21(4), 404-414.
ul Islam, F. M. M., & Lin, M. (2015). Hybrid DVFS scheduling for real-time systems based on reinforcement learning. IEEE Systems Journal, 11(2), 931-940.
Brulé, M. R. (2013, March). Big data in E&P: Real-time adaptive analytics and data-flow architecture. In SPE Digital Energy Conference and Exhibition (pp. SPE-163721). SPE.
Nurse, E., Mashford, B. S., Yepes, A. J., Kiral-Kornek, I., Harrer, S., & Freestone, D. R. (2016, May). Decoding EEG and LFP signals using deep learning: heading TrueNorth. In Proceedings of the ACM international conference on computing frontiers (pp. 259-266).
Wu, D., Liu, S., Zhang, L., Terpenny, J., Gao, R. X., Kurfess, T., & Guzzo, J. A. (2017). A fog computing-based framework for process monitoring and prognosis in cyber-manufacturing. Journal of Manufacturing Systems, 43, 25-34.
Gade, K. R. (2018). Real-Time Analytics: Challenges and Opportunities. Innovative Computer Sciences Journal, 4(1).
Gade, K. R. (2017). Migrations: Challenges and Best Practices for Migrating Legacy Systems to Cloud-Based Platforms. Innovative Computer Sciences Journal, 3(1).
Komandla, V. Transforming Financial Interactions: Best Practices for Mobile Banking App Design and Functionality to Boost User Engagement and Satisfaction.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
License Terms
Ownership and Licensing:
Authors of research papers submitted to Distributed Learning and Broad Applications in Scientific Research retain the copyright of their work while granting the journal certain rights. Authors maintain ownership of the copyright and have granted the journal a right of first publication. Simultaneously, authors agree to license their research papers under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) License.
License Permissions:
Under the CC BY-NC-SA 4.0 License, others are permitted to share and adapt the work, as long as proper attribution is given to the authors and acknowledgement is made of the initial publication in the journal. This license allows for the broad dissemination and utilization of research papers.
Additional Distribution Arrangements:
Authors are free to enter into separate contractual arrangements for the non-exclusive distribution of the journal's published version of the work. This may include posting the work to institutional repositories, publishing it in journals or books, or other forms of dissemination. In such cases, authors are requested to acknowledge the initial publication of the work in this journal.
Online Posting:
Authors are encouraged to share their work online, including in institutional repositories, disciplinary repositories, or on their personal websites. This permission applies both prior to and during the submission process to the journal. Online sharing enhances the visibility and accessibility of the research papers.
Responsibility and Liability:
Authors are responsible for ensuring that their research papers do not infringe upon the copyright, privacy, or other rights of any third party. Scientific Research Canada disclaims any liability or responsibility for any copyright infringement or violation of third-party rights in the research papers.
If you have any questions or concerns regarding these license terms, please contact us at editor@dlabi.org.
