Explore the Flow: Concept Summary

Let’s summarize what we just learned in each of the concept videos. Then, we’ll continue with the hands-on lesson where you can apply each concept.

Flow

In DSS, the Flow is the visual representation of how data, recipes, and models work together to move data through an analytical pipeline. The Flow in DSS has an awareness of the relationships and dependencies between datasets in your project. For example, if you make changes to a dataset or recipe, you may choose to dynamically rebuild dependent items upstream or downstream in your Flow to reflect the update.

To enhance interpretability, you can select parts of the Flow and add tags. You can then use tags to decide which parts of the Flow you want to reveal. You can also hide parts of the flow to decrease the overall number of objects that appear on the screen.

Computation Engine

Computation has a cost. In DSS, there are computation strategies that help with reducing this cost. DSS can perform the computation using the DSS engine or push down the computation to external engines. DSS acts as an orchestrator of your connections’ engines, delegating the computation to these connections when possible.

When transforming your dataset, you are actually working with a sample of it. When you are ready to apply transformation steps to the whole dataset, you click Run. Then, you can select a computation engine that best fits the underlying data storage and the operation you are applying to the dataset.

Job View

DSS creates a Job when you build a dataset, run a recipe, or train a model. DSS lets you know when a Job is running and when it completes. To find out more about a Job or debug a Job, you can view the Jobs log. If a Job fails, you can take steps to diagnose or troubleshoot it.