The SEPIDATA developer studio contains project types, item templates, msbuild integration and developer tools to build a datawarehouse solution for the SEPIDATA datawarehouse automation platform. It also includes an integrated runtime engine for rapid iterative development and testing cycles within the Visual Studio environment. Equipped with all the great features of Visual Studio developing and deploying your data warehouse was never so easy and fast.

• Support for Visual Studio 2017.
• Support for SQL Server 2016 and higher.
• Define your datawarehouse objects and structures in JSON.
• Build you application and get warning and validation errors as if you are coding a .NET application.
• Deploy your solution and run and test it directly after you make changes.
• The visual studio embedded intellisense helps you in picking the correct options for settings on specific DWH objects.
• Build your DWH solutions by creating projects that reference each-other like normal application development.
• Get errors and warnings when you build your DWH solution in Visual Studio.
• The JSON file is validated: does it contain all options and properties for the given object and is it consistent with the rest of the model(s).
• We embedded the rules of the DWH methodologies in our custom build compiler so when you build the solution we protect you from common pitfalls

• Source Model: With this project type you can describe the source system or data model. You can add tables, views, stored procedures and files that are present at your source and where we will retrieve the data from to load it in the staging area.
• Staging Model: With this project type you can define an interface and a target staging area which uses the above described source model. Within the interface you map the structures from your source model to the staging area. Within the staging area you can configure all staging settings: for example if you want to historize the specific table, what type of mutations you would like to detect and much more.

Because our developer studio is build on top of Visual Studio we provide your datawarehouse development team with all the great features of Azure DevOps (on premise or in the cloud). This way you are equipped with a lot of powerfull features for managing the work of your team.

The backlog management capabilities integrate perfectly in Visual Studio and allow you to assign work items so you can relate datawarehouse changes to the backlog items for a perfect audit trail of all changes. This way not only the “how” is visible, but the team and auditors can also see years after a change what the reason was behind a certain change in the datawarehouse.

We use JSON combined with smart wizards to develop your data warehouse solution. JSON is easy for humans to read and write and it is easy for SEPIDATA to parse and generate large pieces of DWH code for you. Within the SEPIDATA developer studio datawarehouse objects like a source table or a staging object are coded or better said “described” in a JSON based format. Every object has its own (by SEPIDATA defined) outline in JSON.

At this moment we do not have any project types that require data transformation because the staging area stores RAW source data. To let you know upfront, when the next project types arrive that do need data transformation (for example the Data Hub or Datamart), you can transform your data by writing SQL code. This SQL script will be embedded within a JSON described transformation object. Around these objects we will provide functionality to parse the script where we will catch the output structure of the query and describe the output structure in a JSON format for you, we will even grab the lineage out of the scripts and add this to the JSON described object. This way you can focus on what’s important: understanding your customers data needs!

Within our developer studio we provide the usual features around coding and compiling. You can build a specific project or the whole solution. You will receive warnings, errors and informational messages that come from our solution compiler. This way you get feedback on the quality of the things that have been developed. We will keep on embedding the rules of the different DWH methodologies in our compiler so we can warn you about possible bad choices or mistakes that will fail your production DWH loads.

We also use the Project References. The staging area project type will always be based on a source model project. Our software will add the project reference so our compiler understands the build dependencies between projects.

We have intellisense implemented to help you choosing options for certain properties of DWH objects. Off course code completion and all the great shortcuts and coding features that are standard in visual studio can be used and will make the life of your developers a lot easier and more productive!

We claim to have the best version control support of all DWH software / ETL tool providers. Your team is free to choose for GIT, TFS as repositories and we support Azure DevOps in the cloud but also the DevOps Server on premise.

When you choose for a TFS repository, you can use the exclusive locking (with Team Foundation based source control). Or if you are more advanced you can go for a GIT repository allowing you to code on a local repository syncing all the changes.

If you require you can also add check-in notes and policies to your project for mandatory documentation of changes. To enhance the quality of your development documentation and audit trail.

Please visit this page of Microsoft on comparing TFVC and GIT. Do you want to learn more about git please visit this site of Microsoft. If you want to know how Microsoft uses GIT please visit this site.

We have strategically chosen the JSON format for coding the DWH solution. This allows you to branch and merge code from different code lines (a.k.a. work with multiple development teams and isolate changes to your datawarehouse). This is where we stand out with regards to the competition, because none of our competitors can provide these features because they have a graphical way of developing data flows were the code behind those flows cannot be merged (like SSIS, Informatica PowerCenter or Wherescape). Also, much of the vendors work with a self-made source control solutions that are not as advanced as TFS or GIT.

This way we allow your company to develop your datawarehouse like professional software development is done. Choose your branching & merging strategy like development isolation, feature isolation and many more.

For example, create a new branch for a source discovery and staging creation on a separate branch and merge it into the DWH when it is ready and tested. Or add a new dimension to the datamart as a feature branch. You will be able to keep doing this also in future project types we are going to add (Data Hub, Datamart etc)

To learn more about branching & merging, please visit this site of Microsoft on DevOps.

Within the developer studio you can create a release build of the datawarehouse solution. This will result in a solution package which can be deployed to the SEPIDATA application server via the REST-API that is running on the application server.

We provide some out of the box example scripts in PowerShell that can be used to deploy a release package and call the REST-API functions. This way you can automate your release deployments.

The SEPIDATA application server will use the automatic upgrade instructions for the new version of your staging project and execute the database upgrade for the staging databases. Operators do not need to execute manual scripts when a new version of the datawarehouse is deployed.