General best practices:
- Project Structure: Generally, each developer in a team has his own preferences of project structure. Which makes it difficult to understand the project for other developers and the same structure for all projects in account brings the consistency, helps fast onboarding new developers into the team. So, it is always important to align the project structure within the team. I prefer the presentation domain data layer for data pipelines.
- Write tests: Automation tests are really important in development. Which will give the quick feedback of code changes and reduces the percentage of bugs in production. There are different types of tests like unit, integration, component and e2e tests. So, take some time and come up with a testing strategy for your projects. Also a good amount of test coverage is important.
Python specific practices:
- Virtual environments: Virtual environment(venv) elements the python packages version conflicts in local development setups. I prefer using the pipenv tool for packaging. It eliminates the old fashion of maintaining the package versions manually in the recruitment text file.
- Linter: Because of python giving a lot of flexibility, it is also easy to write messy code and unconventional coding practices. Identifying these manually is really tough. So, using a linter really helps us to improve coding standards. I prefer using the pylint tool.
- Typecheck: Adding the types to the code improves the readability of the program and can be used by IDE to raise warnings. I prefer using the mypy tool.
- Formatter: Formatting the code is important, it improves the readability and makes code beautiful :). Auto formatting improves the developer productivity by allowing them to focus on logic instead of formatting. I prefer using the autopep8 tool.
- Last and very important one is you need to have a discussion with your whole team and have common understanding on the above practices.
