Delivering Project & Product Management as a Service

How to manage R&D activity in AI?

Research and Development (R&D) has two flavors:

  1. Basic research – Experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundations of AI – Like developing the LLaMA model by Meta.
  2. Applied Research – Investigation directed primarily towards a specific practical aim or objective. Examples include developing AI applications for specific industries, such as healthcare or finance, or fine-tuning an existing model to specific needs.

 

Team building and organization:

  • Centralized AI unit – Establish a centralized AI unit or center of excellence to provide guidance, best practices, and support to AI projects across the organization.
  • Interdisciplinary teams – Best practice is assembling cross-functional teams consisting of AI researchers, data scientists, domain experts from the relevant fields, engineers, and product managers to foster collaboration and fast decision making within the team.
  • Skill development and training – Invest in ongoing skill development and training programs to keep the workforce up to date with the latest AI technologies and best practices.
  • Partnership ecosystem – Foster partnerships with academic institutions, research organizations, and technology vendors to access cutting-edge research, talent, and tools.

 

To do AI you need tools:

  • High-performance computing infrastructure to deal with large datasets and computations involving GPU and TPU (google) – This is mostly provided by cloud services like AWS, GCP and Azure.
  • Data engineering tools to store, process and analyze data.
  • AI frameworks and libraries – Like TensorFlow, PyTorch, and scikit-learn to build on the shoulders of others.
  • Collaboration and version control tools – Notebooks and using platforms like GitHub and GitLab enable collaboration, code sharing, and version control.

 

Once you have the tools you need to plan how to operate them:

  • Data governance – Data rules (and regulated), so establish data governance policies and procedures to ensure data quality, security, and compliance with regulations.
  • Agile development – Agile became standard in software, and its principles like iterative and incremental development of AI prototypes and solutions should be used as much as possible.
  • Continuous integration and deployment (CI/CD) – When moving to production, the “production line” should be automated as much as possible, so changes are fast and agile.
  • Model monitoring and maintenance – Any deployed AI model should be monitored for performance and maintenance when accuracy is reduced due to natural data changes.

 

Stopping rules or KPIs. R&D should produce results with value, those should be traced with Key Performance Indicators like:

  • Research output – For basic research count the number of publications and impact factor.
  • Innovation pipeline – Count the number of AI projects in various stages of development, from ideation to deployment, to ensure a healthy innovation pipeline.
  • Average time to market – Measure the time taken from project initiation to the deployment of AI solutions to drive efficiency and competitiveness.
  • Number of patents – Patents are the gates to commercialization, so in applied research it’s a good measure.
  • Return on investment (ROI) – Evaluate the financial impact of AI projects, considering factors such as cost savings, revenue generation, and operational efficiency improvements.
  • Collaboration and knowledge sharing – Assess the level of collaboration and knowledge sharing within and across teams through metrics like the number of cross-functional projects, internal workshops, and knowledge-sharing sessions.

 

This will make a robust framework for an organizational AI function.