Best practices in developing a tool/software [@hastings2014ten]
The gap
Scientific research is increasingly relying on computing as a core platform, while the growing scale, complexity, variety, and accessibility of datasets across many formats increase the need for well-designed, efficient, and easy-to-maintain software and tools. Establishing good software development practices is essential to improve code quality, reduce errors, support teamwork, lower maintenance costs, and ensure software remains reliable, reusable, and sustainable over time.
Recommendations
Keep it clean and simple
Starting small and simple has proven to be a better way to begin with an idea. Overengineering or underengineering can lead to chaos and confusion. It is always recommended to start small and build as one proceeds. Do the simplest thing that could possibly work, and then double-check it really does work.
Test along each step of the way
In any project, testing at every step avoids mistakes and repetitive work at the end. Thus, designing test cases at the very beginning of the project serves as a check that each step of the project undergoes. These are called unit tests and are highly recommended for implementation during the planning stage of a software-based project. These tests should consider all the borderline test cases to make the software/tool efficient.
Careful when pasting the data
When the scenarios or problem statements are similar, it is common to use already established work in the form of code or content. This could result in line duplication or leave bugs that will be hard to fix in the end. Automated tools, such as Simian, can help to detect and fix duplication in existing codebases.
Modular design for workflow
Adopt a modular design in which software is built from separate components that work together to provide the system’s overall functionality. Each module should expose a clear public interface while hiding its internal implementation details. Developers should write and test code against this interface rather than the underlying implementation, so that internal changes can be made without affecting other parts of the software. Shared application programming interfaces can also support multiple implementations.
Before creating new functionality, carefully evaluate existing modules and libraries. Reusing well-established solutions is often more efficient than developing new ones from scratch, even if a library offers more features than are immediately needed.
Use cases
Involving users in the early stage of development is always recommended. This helps create a tool that can cater to users’ needs. Involving users early on in open projects is a very common practice. If the data is confidential/sensitive, a beta session can be created for a specific target group to discuss.
Know the limitations of the tool
Developers should be careful not to add unnecessary features simply because users request them or because they seem useful at the time. It is important to distinguish core requirements from optional enhancements and to prioritize features with input from a broad range of stakeholders. Extra additions can appear harmless in isolation, but repeated small changes across development cycles can gradually lead to unnecessary complexity and feature creep.
Documenting every step of the way
Clear documentation is essential for making software easier to understand, maintain, and transfer to other developers. Comments should explain complex logic and describe public methods and interfaces, but they do not need to restate every line of code. The goal is to provide useful context that will help both future collaborators and your future self.
Optimize as the last step
Avoid premature optimization. Even when performance matters, it is often difficult to know where the real bottlenecks will appear until the software is used with realistic data and in its intended environment. The priority should be to build correct, reliable functionality first, then gradually improve performance through measurement and testing. Continuous runtime evaluation, supported by unit tests, helps ensure that optimization does not break expected behavior.
Favor evolution over revolution
As software ages, maintaining it can become more challenging, so it is important to regularly review and improve the codebase. However, rewriting an entire system from scratch should generally be a last resort, unless the software is very small or no other practical solution exists. Incremental improvement is usually more realistic and sustainable than a full rebuild, which may never be completed.
Use version control consistently
A reliable version control system, such as Git, together with a central repository, helps manage changes and supports collaboration. Changes should be committed early and often, not only during major refactoring, so that development history remains clear and recoverable.