Neural operators (e.g., Fourier Neural Operator, DeepONet) learn mappings between function spaces and increasingly replace expensive numerical simulations. In practice, however, the real process always deviates from the simulation, and real measurements are scarce, noisy, and often only partially available. Within this project, we investigate how an operator pretrained on simulation data can be corrected using few, partially covered observations of the real system (residual or delta modelling, sim-to-real transfer). This thesis lays the methodological foundation on public benchmark problems of operator learning.
First, a neural operator is to be trained on an established benchmark problem (e.g., Darcy flow). Subsequently, a synthetic sim-to-real gap is to be created, for instance through perturbed coefficients or additional terms in the reference model. The core of the thesis is the implementation and comparison of methods for residual correction of the pretrained operator from few observations, where observations are only partially covered due to masking. The relationship between observation coverage, number of measurements, noise level, and achievable correction quality is to be quantified systematically. As a baseline, at least Gaussian process regression is to be included. The resulting evaluation environment should be built for reuse.
Solid Python skills and experience with a deep learning framework (PyTorch preferred); fundamentals of machine learning; interest in numerical simulation and scientific computing. Prior knowledge of neural operators or Gaussian processes is helpful but not required.
Sebastian Baum
♦
In deep drawing of cylindrical parts, periodic shape deviations known as earing form at the rim. In a production line, the rim contour h(φ) can often only be captured on partial arcs, and depending on the measurement position, the angular orientation of the part relative to the tool is unknown. Within this project, this poses a well-defined estimation problem: how well can a periodic contour be reconstructed from partial, noisy observations, and under which conditions can the unknown rotation be estimated from the characteristic, unequal expression of the ears themselves?
Based on synthetic contour data (parametric test contours and contours from preliminary finite element runs), an estimation method for reconstructing h(φ) from partial arcs is to be implemented and its conditioning analysed as a function of coverage. Subsequently, registration of the unknown rotation via matching against the asymmetric earing signature is to be implemented (e.g., cross-correlation), and its success probability quantified systematically as a function of asymmetry level, noise level, and coverage. The results feed directly into the selection of the measurement position in the project. Optionally (research project), the rotation can alternatively be treated probabilistically as a latent variable.
Solid Python skills (NumPy/SciPy); fundamentals of signal processing or estimation theory (Fourier analysis, regression); a structured, mathematically rigorous way of working. Knowledge of forming technology is not required.
Sebastian Baum
♦
Modular process plants, such as the offshore PtX plant developed in the H2Mare project, are highly complex systems. Operating personnel require extensive process and plant knowledge in order to control the process safely and to respond appropriately in the event of a fault. However, the necessary information is distributed across a wide range of heterogeneous sources, including operating manuals, signal and alarm lists, and maintenance documentation, all of which have to be searched manually. Experience from offshore test campaigns shows that considerable potential for improvement in efficiency and reliability exists in dealing with alarm floods, system faults, and diagnostic processes.
A particular feature of modular plants according to VDI/VDE/NAMUR 2658 is that plant structure, service descriptions, state machines, messages, and OPC UA addressing are already available in machine readable form within the Module Type Package (MTP). This source has so far remained unused by document centred assistance approaches.
Generative Artificial Intelligence (GenAI) opens up new possibilities in this context. Operator assistance systems can combine live process data, alarm histories, formalised module descriptions, and experiential knowledge in order to support operating personnel in diagnosis, decision making, and maintenance.
The aim of this thesis is to design and prototypically implement a GenAI based operator assistance system that supports operating personnel in monitoring, diagnosis, maintenance, and fault handling. Drawing on live process data, alarm messages, MTP module descriptions, and plant documentation, the system is intended to provide context specific recommendations for action, fault analyses, and decision support.
The objective is a general, transferable architecture for GenAI assistance systems in modular process plants based on OPC UA and MTP. The concept is to build on LLM and RAG technologies, to access process and knowledge sources selectively through a tool interface, and to use the machine readable MTP descriptions as the primary source of plant structure and data point semantics. Since the target environment is a largely unmanned offshore platform with limited network connectivity, the system is to be operated entirely locally on edge hardware.
Development and validation are carried out on a modular PtX model plant with a SIMATIC PCS neo process control system at IAS.
Your tasks:
- Familiarisation with GenAI approaches (LLMs, RAG) and with modular automation according to MTP
- Design of a modular, extensible layered architecture
- Connection to the process control system and development of the knowledge base
- Prototypical implementation of a demonstrator including a user interface
- Evaluation with regard to practical suitability, robustness, and transferability
- Independent and methodical approach to research
- Basic knowledge of the structure and function of process control systems in plant automation
- Interest in generative AI and its application in industrial environments
- Initial experience with LLMs and RAG systems is an advantage
- Familiarity with standards such as OPC UA or MTP is an advantage
- Programming skills in Python are an advantage
Peter Pfeiffer
♦
For robots to be used in automation technology, they must be able to make reliable and safe decisions even in unforeseen situations. Different safety mechanisms must be implemented for different types of robots and applications: While a mobile robot can be shut down and brought to a stop in an emergency, shutting down a humanoid robot could potentially damage it.
To define the optimal behavior for different types of robots in unsolvable or safety-critical situations, a classification (taxonomy) of the various cases is required.
The goal of this research project is to develop a taxonomy that can be used to classify the desired behavior of robots in unsolvable or critical situations. The study will examine various types of robots (humanoid, robotic arm, mobile) as well as different applications.
A literature review will examine existing taxonomies before a new taxonomy is developed that assigns different target behaviors to robot types and domains. Possible target behaviors when confronted with an impossible task could include, for example, achieving the next best goal, shutting down, remaining stationary, or assuming a neutral position.
In addition, a concept for a dataset that reflects this taxonomy will be outlined. To this end, impossible scenarios will be designed, and the various target behaviors will be mapped within them.
- Knowledge on how to perform a literature analysis
- AI-Experience (LLM, VLM)
Nico Wilhelm
♦
A Franka Emika robot arm does a pick-and-place sorting task. Its control, path planning
and gripping already work. A behavior tree runs the task and calls skills. All of it runs in a
Docker container. The arm has no camera, so it cannot tell what is on the table or where.
This project adds one.
An Intel RealSense D405 depth camera is mounted on the arm's flange. It must be
calibrated against the arm, find the objects in the workspace, and report where they are
in the robot's own coordinate system. A second step then takes those objects and works
out how the gripper should grasp them. The arm then executes the grasps, so that the
whole chain can be measured end to end.
Goal:
Build a perception stack that answers two questions:
1. What objects are on the table, and where are they?
2. For a given object, what gripper pose would grasp it?
The stack runs in its own Docker container on a Jetson Orin AGX. A baseline Docker image
is provided. It must plug into the existing robot software through a clear interface.
Neither the gripping nor the motion control is implemented in this project, both already
exist. Training neural networks is not part of it either. The student reviews what exists,
picks suitable networks, and builds the software around them.
1. Literature research: Review object detection networks and grasp pose
generation networks. Select the ones to use in the stack and justify the choice.
Cover the practical criteria as well as the published results.
2. Bring up the camera in ROS 2: Get the RealSense publishing color and depth
images inside the container.
3. Hand-eye calibration: Determine where the camera sits relative to the arm's
flange using the existing software packages. Document the procedure so it can be
repeated.
4. Build the Detection Pipeline: Detect and segment the objects in the camera
image, producing a pixel mask for each. Convert each mask into 3D points in the
robot's base frame using the depth image, the camera intrinsics and the
calibration. Wrap this as a ROS 2 action the behavior tree can call. The action
takes a picture, runs the detector and returns the result. It does not move the arm.
Define an interface that passes the scene and the per-object information to the
next module.
5. Build the grasp node and its action: Implement a second node with its own
action. It receives the detection result and returns grasp poses, using the network
selected in task 1.
6. Evaluation:
a. Calibration consistency: Place the calibration board somewhere fixed.
View it from at least 15 arm poses that dier in rotation, not just position.
For each pose, compute the board's position in the robot's base frame. If
the calibration is good, all the answers agree. Report on the standard
deviation.
b. Detection rate: Using a fixed test set of objects, report how often objects
are found, how often something is reported that is not there, and how much
an object's reported position varies across repeated measurements from
different viewpoints.
c. Grasp success rate. Using the same test set, let the arm attempt the
generated grasps. A grasp counts as successful if the object is lifted clear
of the table and held. Report the success rate per object and break the
failures down by cause.
Optional Task
A. Measure Calibration Accuracy: 6.1 measures precision, not accuracy: a
calibration can be consistently wrong. Measuring accuracy needs an independent
reference. Devise
Expected at the start:
- Python
- Basic Linux command line
- Willingness to work with hardware and to debug problems that are not in software
Helpful, but can be learned during the project:
- ROS 2: nodes, topics, actions, tf2
- Docker
- Camera models, depth images and point clouds
- Using pretrained neural networks
Georgios Katranis
♦