Notebooks

In Datalore you can create Kotlin, Scala, and R notebooks. You can use magics to install packages, and when writing code, you’ll get code completion.

Add native SQL cells to query your database connections. In addition to SQL syntax-highlighting, you also get code completion based on the introspected database tables. The query result is automatically transferred to a pandas DataFrame and you can continue working on the dataset in Python.

Use SQL cells to easily query 2D DataFrames and CSV files from attached documents, just like you would with databases. Simply browse through your notebook’s DataFrames, pick one, and use it as your source for SQL cells. With this feature, you can merge data from various sources into a single DataFrame using SQL or break down complex queries into a sequence of SQL cells.

Embed your variables in Markdown cells with double curly brackets. Variables will dynamically convert to their live values within your text.

Apply filtering and sorting to Pandas DataFrames and SQL query results directly in the cell output. Select the columns to display, sort the dataset by a specific column, filter based on “equals” and “contains” expressions, and easily jump to the top or bottom of the dataset. After you complete the filtering and sorting, use the Export to code cell option to generate the Pandas code snippet and make the table view reproducible.

You no longer need to download CSV files to make a set of edits in a DataFrame. You can simply edit the content of cells inside interactive tables and click Export to code to reproduce the result in the notebook.

With one click, you can get essential descriptive statistics for a DataFrame inside a separate Statistics tab. For categorical columns, you will see the distribution of values, and for numerical columns, Datalore will calculate the min, max, median, standard deviation, percentiles, and also highlight the percentage of zeros and outliers.

Add interactive inputs such as dropdowns, sliders, text inputs, and date cells inside your notebooks, and use the input values as variables inside your code. Present the visualizations with Chart cells and highlight specific numbers within Metric cells.

Report and notebook owners can now enable collaborators to upload CSV, TXT, or image files from their local machines. Set file types and size limits to seamlessly incorporate file uploading into your workflow.

Export DataFrames to existing tables in a database directly from your notebook. Customize the export by picking the DataFrame, target database, schema, and table. You can also take advantage of the scheduling feature and automate the exports.

Datalore supports IPyWidgets, the classic Jupyter widgets framework. Add interactive controls with Python code, combine multiple widgets in one cell output, and use the selection as a variable in the following parts of your notebook.

Open CSV and TSV files from the Attached data tab in a separate tab inside Datalore’s editor. Sort the column values and paginate the file contents.

Create and edit CSV and TSV files right inside the Datalore editor. You can start from scratch and create a new file or you can edit the contents of an existing one.

Open Terminal windows inside the editor, execute .py scripts, and access the agent, environment, and file system using standard bash commands.

Browse notebook variables and built-in parameter values from one place.

Create custom history checkpoints that enable you to revert the changes at any time using the history tool. When browsing a checkpoint, you will see the difference between the current version of the notebook and the one that is selected.

Use scheduling to run your notebooks on an hourly, daily, weekly, or monthly basis and deliver regular updates to published reports. Choose the schedule parameters from the user interface or use the CRON string. Notify notebook collaborators upon successful or failed runs via email.

Specify and manage multiple schedules for a single notebook through the user interface. This feature allows you to create custom schedules and run your notebook hourly, daily, weekly, or on specific dates. With the ability to set up different schedules based on your unique needs, you can achieve more efficient resource allocation and tailor the code execution timings to your project’s demands.

It is now possible to trigger running a Datalore notebook or republishing a report with an API call. This feature is an addition to scheduled runs, which allows you to trigger rerunning a notebook on-demand from external apps and internal Datalore notebooks. It is also possible to view the run results from the Scheduled run menu. More information on how to use the API can be found here.

For R notebooks, you can now install packages from public and private R package repositories supported by install.packages inside the Environment manager tab. Using Environment manager helps keep your environment configuration persistent throughout the notebook runs. It's possible to configure a custom repo by creating an .Rprofile file in init.sh or a custom agent image.

While the conda installation remains the default in the cloud version, Enterprise customers can configure a non-conda custom base environment with the R kernel. This will lead to an absence of conda packages in the Environment manager search results. You can find an example of such an installation here.

Locate specific code sections or find the necessary information in all notebooks across your workspaces. In addition to searching for notebook names, you can now search for variable names and content. You will see your query highlighted in the search results.