I have been using an adapted version of the SharePoint data loader to extract files from SharePoint sites, but now I need to add the capability to extract SharePoint list items instead of files in drives. The logic for doing this is so different that it should be in a completely separate data loader, but the collection of SharePoint APIs I’m using is very similar. So, ideally, I would like to extract my SharePoint API client to a separate module that can be shared between the data loader modules.
I tried doing this locally, and it seemed to run without any issues. Instead of all the code being in one file that is passed to the
--source-script argument of
squirro_data_load, I put my API client in a
sharepoint_client.py file in the same directory, put the data loader class (i.e. the one inheriting
DataSource) in a
dataloader.py file that imports the SharePoint client class, and passed the path to the latter as my
However, I have never seen any Squirro data loaders do things this way, even though factoring code into modules is generally best practice and some data loaders can be several hundred lines of code. Are there any Squirro-specific reasons why it’s problematic to have multiple Python files in your data loader directory and import them locally, or gotchas to look out for?