Data Scientist - Model Evaluation

Introduction
Rise to meet the challenge of solutioning the world's problems. At IBM, work is more than a job - it's a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you've never thought possible. Are you ready to lead in this new era of technology and solve some of the world's most challenging problems? If so, lets talk.

Your Role and Responsibilities
We are looking for Data Scientist to work on dynamically evolving IBM watsonx.ai offering from Kraków (Poland) office in product area of Model Evaluation.

In this position, you'll be immersed in the forefront of the AI revolution. Your duties will include critically assessing generative AI models and processes/enhancements related to them, establishing quality benchmarks, and challenging models' capabilities.

Your role and responsibility will encompass:

• Critically assessing and pushing the boundaries of generative AI models and tuning of such models, focusing on LLM models
• Establishing and upholding stringent quality benchmarks
• Collaborate with the research team to develop state-of-the-art NLP solutions utilizing LLM models

A successful candidate for this role should demonstrate:

• An intrinsic motivation, coupled with an insatiable appetite for continuous learning
• Proficiency in crafting and sustaining tools to amplify productivity
• Robust analytical capabilities and a keen problem-solving acumen
• A tenacity to assume responsibility, driving problems to their resolution and tasks to fruition
• Proficiency in collaborating seamlessly within diverse team dynamics

For an overview of the product, see https://www.ibm.com/products/watsonx-ai

Required Technical and Professional Expertise

• Professional experience in AI/ML model development and evaluation with solid foundation in linear algebra, statistics, and optimization
• Familiar with automated ML models evaluations techniques, including Large Language Models (LLMs)
• Familiar with advanced model training methodologies including prompt tuning and fine-tuning with foundational models
• Proficient in Python, with experience in coding and debugging in Python and Jupyter Notebook
• Demonstrated ability to critically assess and challenge AI (including LLM) models to ensure optimal performance
• Fluent in written and spoken English

Preferred Technical and Professional Expertise

• Expertise & up-to-date knowledge of advances in Foundation Models area, focusing on Large Language and Multimodal Models
• Design, develop NLP models to extract insights and information from unstructured data
• Background in developing and implementing test automation frameworks and strategies
• Experience with cloud environments