An agile approach to Data Science

Most Professional Services engagements with OpenText™ follow a traditional design, built, test and deploy project methodology.  OpenText software is well suited for the waterfall project…

Marc St-Pierre profile picture

Marc St-Pierre

October 4, 20225 minutes read

Most Professional Services engagements with OpenText™ follow a traditional design, built, test and deploy project methodology.  OpenText software is well suited for the waterfall project model.  A notable exception is OpenText™ Magellan™ and our Data Science projects.  In these cases, customers can expect an approach which simply adds refinement iterations to the build phase or otherwise, and full adoption of a more agile-like methodology, named OpenText Data Science Methodology.

Any data science project requires 3 important ingredients: Data, People and Technology.   With any element missing or being deficient, a data science project will be doomed, or even fail to get off the ground.  Finding the required data and the subject matter expertise is a complex exercise.  As this step may need to be tested and repeated many times over the lifetime of the project, a data science project must be approached with an agile or agile-like methodology.

What is Agile software development?

The History of Agile

A recent report by Wellingtone claims that only 29% of the organizations complete most or all of their projects on time, only 43% of the organizations complete most or all of their projects on budget and only 40% of organizations deliver the full benefits of most of their projects. Software projects are challenging for many reasons

In 2001, a group of thought leaders met at a ski resort in the Wasatch mountains, in Utah, to talk about the challenges of software development projects and how these projects could be more successful and efficient. They documented 4 key values in the Agile Manifesto to help project managers and developers build better software faster. 

The Agile Manifesto’s four key values:

1. Individuals and interactions

The Agile approach states that projects should be built around motivated individuals. The team can be motivated by a clear product vision and mission statement. It is crucial that the team remains focused on completing the tasks while working on the project. Team members perform best when they feel their contributions are important, valued and trusted. Face-to-face conversations are the most effective way to communicate. Communication via email is not as effective as it might take several email exchanges to sort out a simple issue that could be addressed with a short face-to-face conversation. 

2. Working software

According to the Agile Manifesto, product improvement is the main indicator of progress. Delivering valuable software to stakeholders early and frequently is more important than demonstrating progress through status reports. Having frequent releases of working software allows stakeholders to provide valuable feedback and lowers the risk of not meeting the needs of customers. 

3. Customer collaboration

Regular communication between the customer and developers is another critical key to building valuable software. Overthinking and over analyzing the requirements at the beginning of the project can lead to analysis paralysis and missed opportunities.

In Agile, the team makes decisions relatively quickly based on available information and adjusts the plan during the project based on regular communication between developers and the customer. The business value of features is determined by the customer, while the effort required to develop these features is provided by the developers.  This approach ensures that the team is always working on the right tasks and the end result fulfills the customer’s needs.

4. Responding to change

Agile means the ability to move quickly and easily. It also refers to flexibility and openness to change. The Agile philosophy acknowledges the fact that in our world things change constantly. According to Heraclitus and Plato, change is the only constant in life. 

Agile accepts and welcomes the lack of certainty and follows informal processes when requirements change. An Agile team must be able to react quickly to changing requirements, environments and shifting trends. 

When should you use Agile?

The Agile approach helps to deal with rapidly changing environments, uncertainty, complex solutions, emerging technologies, and ambiguous requirements. While Agile doesn’t eliminate uncertainty, it reduces risks and promises better outcomes in unpredictable and complex scenarios by delivering high-value products early and frequently and reviewing the result with the customer on a regular basis. 

OpenText expands this Agile approach into our Data Science Project methodology, adding focus to key areas:

  • Business understanding
  • Data understanding
  • Data preparation
  • Modeling
  • Evaluation
  • Deployment
  • And repeat..
OpenText Data Science Methodology

In our next blog, we will look more in-depth at the elements of the OpenText methodology and how it’s applied in engagements like Idea to Insight in 30 days.

Until then, discover how OpenText helps organizations become more agile by implementing business ready AI and Analytics solutions and providing Data Science, NLP and Knowledge Management services to solve today’s complex business challenges.  Visit AI & Analytics Services | OpenText for more information.


Contributor: George Berlak is the AI and Analytics Solution Development Manager for OpenText Professional Services. He is a certified Artificial Intelligence, Analytics and Data Science professional and leads a team of architects and engineers to design and build strategic AI, Analytics and Natural Language Processing solutions.

Share this post

Share this post to x. Share to linkedin. Mail to
Marc St-Pierre avatar image

Marc St-Pierre

Marc is VP of Consulting Services for the Security + Artificial Intelligence + Linguistics & Translation practice. For more than 15 years, Marc has led services groups specialized in advanced and emerging technologies. He has lectured on semantic technologies and lead solution development such as Ai-Augmented Voice of the Customer and Magellan Search+.

See all posts

More from the author

Cybersecurity Services combat an APT with NDR

Cybersecurity Services combat an APT with NDR

Attackers linked to Iran and China are actively targeting critical infrastructure.  Both the U.S. Environmental Protection Agency and National Security Agency have requested that each…

March 28, 2024 4 minutes read
Strengthening Higher Education Institutions against evolving cyberthreats

Strengthening Higher Education Institutions against evolving cyberthreats

As cyberthreats continue to evolve, it is crucial for higher education institutions and universities to be vigilant.  Enforcing security strategies prudently designed to safeguard digital…

January 24, 2024 4 minutes read
Strengthening cyber resilience

Strengthening cyber resilience

Cyberattacks are on track to cause $10.5 trillion a year in damage by 2025. That’s a 300 percent increase from 2015 levels. A robust cybersecurity…

December 19, 2023 4 minutes read

Stay in the loop!

Get our most popular content delivered monthly to your inbox.