Despite its potential, the adoption of Artificial Intelligence (through advances in language models, computer vision, and generative models such as GPT-4, LLaMA, DALL-E, or CLIP) in industrial environments still presents significant barriers. Current visual inspection and anomaly detection solutions rely on large volumes of labeled data, which entails high costs. Furthermore, available interfaces do not allow for natural interaction in human language, limiting their usability in the plant floor.
In this context, the IKUN project, comprised of the technology centers members of the BRTA VICOMTECH (leader), TECNALIA, IKERLAN, TEKNIKER, and AZTERLAN; the University of the Basque Country (UPV/EHU); the IKOR TECHNOLOGY CENTER business R&D unit; and the intermediary agent IMH Campus, was created with the aim of overcoming these challenges by adapting Large Multimodal Models (LMMs) to the industrial context, laying the foundations for a smarter, more autonomous, and connected industry. Thus, IKUN addresses several key challenges for the integration of LMMs in industrial environments:
- Creation of multimodal industrial datasets, which serve as a basis for adapting these models to the specific production domain.
- Research into LMM adaptation techniques, aimed at ensuring robust, explainable, and reliable models under real-life operating conditions.
- Generation of synthetic data, including industrial images and time series, to train advanced visual inspection and anomaly detection systems without the need for large volumes of real data.
- Development of multimodal conversational interfaces, allowing operators to interact naturally (text, voice, image) with systems, facilitating access to technical knowledge and dashboards.
The project is based on a practical and progressive approach that ranges from the definition of pilot projects and the collection of real-life industrial data to the adaptation of multimodal models to the production environment. Specific models for image and time series will be developed, and conversational interfaces that improve the interaction between operators and systems will be explored. Finally, all developments will be validated in real-life industrial environments to ensure their applicability and transferability.
Expected Results
- The creation of unprecedented multimodal industrial datasets, key to the training and evaluation of new models.
- The generation of high-quality synthetic industrial images and time series that reduce dependence on real-life data.
- The design of intelligent conversational assistants capable of assisting operators through text, voice, and images, both on the production line and through documentary sources (manuals, technical reports).
- The development of validated prototypes with high transfer potential to the local industrial ecosystem.
The IKUN project “Large Multimodal Models for Quality Assurance and Supporting Industry workers in Smart Industry” is funded through the ELKARTEK 2024 program (KK_2024 00064) of the Basque Government.
