Author: Lougarou

In this post we look into:

1. Connecting to ESDB
2. Writing events
3. Reading the events
4. Deserializing the events
5. Sharing the events with Javascript code
6. Visualizing the events with d3js.

You can check the code at: https://gist.github.com/Lougarou/7da6e3b69ea803b263037b1b7a86ab81

How to create a Polyglot Notebook project in VS Code

Step 1: Install Visual Studio Code

Step 2: Install the extension and .NET 7 SDK https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.dotnet-interactive-vscode

Step 3: From Visual Studio code: Click Help->Show All Commands->Polyglot Notebook create default notebook as shown below:

Then choose Choose .ipyb and then F#.

You are now ready to write interactive code in multiple languages as we will see further. By the way Visual Studio Code has likely installed .NET interactive in background.

How to specify different languages in Polyglot Notebooks

You can either write #!fsharp at the start of the cell or you can click and choose bottom right as show in the screenshot. Upon clicking on it you will find all the Notebook Kernels available to run your cells.

For example:

How do you install NuGet Libraries in Polyglot Notebooks?

Next we need to import our EventStore gRPC Client libaries like we did in Using F# and EventStoreDB’s gRPC client is easy

To do that we need to add a special #r and then details about our NuGet package. So for us the following:

#r "nuget: EventStore.Client.Grpc, 23.0.0"
#r "nuget: EventStore.Client.Grpc.Streams, 23.0.0"

If you are not sure what to write, the web page of the NuGet package will help you. Click on the Script & Interactive tab:

If you then click on the Play button of the cell you will see that VS Code will try to install the packages:

Once this is done, click the +Code button to create a new cell.

Write Events To ESDB with FSharp

Let’s start by writing events. We already covered that in Using F# and EventStoreDB’s gRPC client is easy. We need to create a list of events and send them to ESDB. We can do that as follows:

open EventStore.Client
open System.Collections.Generic

let client = EventStoreClientSettings.Create "esdb://127.0.0.1:2113?tls=false"
             |> EventStoreClient

let streamName = "house_price_changes"
let eventsList = List.init 10 (fun index ->
    EventData(
        Uuid.NewUuid(),
        "rent",
        ReadOnlyMemory<byte>(Encoding.UTF8.GetBytes("{\"USD\":"+ string(Random().Next(1000,2000))+", \"year\":"+string(2000+index)+"}"))
        )
    )
client.AppendToStreamAsync(streamName, StreamState.Any, eventsList).Wait() //assuming stream doesn't exist

We are creating JSON events, you can use System.Text.Json to serialize types if you want. Later we will cover how to Deserialize your JSON events in FSharp.

Adding D3.js code

Create a new cell, this time put the kernel as Javascript or add #!javascript at the start. In the code, we will create a bar plot. You can do more complex visualizations but the underlying principles are the same.

configuredRequire = (require.config({
    paths: {
        d3: 'https://cdn.jsdelivr.net/npm/[email protected]/dist/d3.min'
    },
}) || require);

var caller = null;
//from: https://github.com/dotnet/interactive/blob/main/docs/javascript-overview.md
plot = function (data) {
    configuredRequire(['d3'], d3 => {
        // Call d3 here.
        //from: https://d3-graph-gallery.com/graph/barplot_basic.html
        const margin = {top: 30, right: 30, bottom: 70, left: 60},
        width = 512 - margin.left - margin.right,
        height = 512 - margin.top - margin.bottom;

        // append the svg object to the body of the page
        const svg = d3.select("#my_dataviz")
        .append("svg")
            .attr("width", width + margin.left + margin.right)
            .attr("height", height + margin.top + margin.bottom)
        .append("g")
            .attr("transform", `translate(${margin.left},${margin.top})`);

        // X axis
        const x = d3.scaleBand()
        .range([ 0, width ])
        .domain(data.map(function(d) { return d.year; }).reverse()) //because I was reading backwards
        .padding(0.2);

        svg.append("g")
        .attr("transform", `translate(0, ${height})`)
        .call(d3.axisBottom(x))
        .selectAll("text")
            .attr("transform", "translate(-10,0)rotate(-45)")
            .style("text-anchor", "end");

        // Add Y axis
        const y = d3.scaleLinear()
        .domain([0, 2000])
        .range([ height, 0]);
        svg.append("g")
        .call(d3.axisLeft(y));

        
        svg.selectAll("mybar")
        .data(data)
        .join("rect")
        .attr("x", function(d) { return x(d.year); })
        .attr("y", function(d) { return y(d.USD); })
        .attr("width", x.bandwidth())
        .attr("height", function(d) { return height - y(d.USD); })
        .attr("fill", "#69b3a2")
    });
}

Here we have imported d3.js from an external source, we are using the year as x-axis and USD as y-axis. We have mapped our domains based on the values in our data to the different positions on the x-axis and height of rectangles (our ranges). D3.js is mainly about mapping data to domains to ranges to shapes in SVG.

Note: If you want to add transitions you can check examples here: https://github.com/dotnet/interactive/tree/main/samples/notebooks/polyglot At the time of writing “true” streaming is not supported, you have to use js intervals and call d3 to update your graphs if you want them to change live with data connected with ESDB.

Reading our data and Deserializing JSON to a FSharp type

open System.Collections.Generic
open FSharp.Control
open System.Text.Json //to deserialize JSON event data to house_price_change type

type house_price_change = { USD : int ; year : int }
let eventToPair (resolvedEvent: ResolvedEvent) = //convert resolved event json data to house_price_change
    let JSONEventString = Encoding.UTF8.GetString(resolvedEvent.OriginalEvent.Data.ToArray())
    JsonSerializer.Deserialize<house_price_change> JSONEventString

let priceChanges  = List<house_price_change>() //we will append data that is read from ESDB and share this list with Javascript

client.ReadStreamAsync(Direction.Backwards, 
                       "house_price_changes",
                       StreamPosition.End,
                       10L,
                       true)
    |> TaskSeq.iter (fun event -> (priceChanges.Add (eventToPair <| event)))
    |> Async.AwaitTask
    |> Async.RunSynchronously

Here we create a type house_price_change type house_price_change = { USD : int ; year : int } and we use System.Text.Json to deserialize our event’s JSON data to that type. This happens in our handler in ReadStreamAsync when we call our eventToPair function. In retrospect this would have better been called eventToHousePriceChanges 😅

We could have also used the serialize function to do the reverse when we were appending events.

Anyway, everyone is an expert in retrospect!

Sharing our variable to Javascript and plotting our data

#!html
<div id="my_dataviz"></div>

#!javascript
#!share --from fsharp priceChanges
console.log(priceChanges); //we receive the variable in a nice format
plot(priceChanges);

Finally, we create a cell with first html code to create a div which will hold our graph. Then we tell the cell to start handling javascript code with #!javascript.

More interesting here, we tell the cell take our priceChanges variables which is a list holding a specific type. This works surprisingly well as you will see from the console output:

[{"USD":1806,"year":2009},{"USD":1020,"year":2008},{"USD":1068,"year":2007},{"USD":1540,"year":2006},{"USD":1088,"year":2005},{"USD":1149,"year":2004},{"USD":1540,"year":2003},{"USD":1085,"year":2002},{"USD":1208,"year":2001},{"USD":1256,"year":2000}]

Then we pass that variable to out plot function defined before which will plot our price changes. This should generate a graph like the one below:

That’s it, you can continue to build on top of this. If you add other kernels like Python you could add even add machine learning to your notebooks (have not tried).

If you are confused about any part of this post please feel free to comment!

The most popular client for EventStoreDB is the dotnet client, so naturally, this makes the library easy to use in F# as well. The first part is creating a project.

Disclaimer #1: I am still new to FSharp/F# so it is likely that the code is not as elegant as you would expect.

Disclaimer 2: This uses F# 6

dotnet new console -lang "F#" -o project_folder_name

Next we need to import the following libraries

https://www.nuget.org/packages/EventStore.Client.Grpc

https://www.nuget.org/packages/EventStore.Client.Grpc.Streams

dotnet add package EventStore.Client.Grpc --version 22.0.0
dotnet add package EventStore.Client.Grpc --version 22.0.0

You should be ready to use the library. Let’s get started with connecting to an EventStoreDB instance.

Import the library with:

open EventStore.Client

Then connect to the instance by providing a connection string.

let client = EventStoreClientSettings.Create "esdb://127.0.0.1:2113?tls=false"
             |> EventStoreClient

The code above initialises an EventStoreDB client with a connection string telling the telling to connect to a single EventStoreDB instance running in insecure mode on localhost listening on the default HTTP port 2113.

After that you have to create a list of events to write. You can do that as follows:

let streamName = "sample_stream"
let eventData = EventData(
                    Uuid.NewUuid(),
                    "event_type",
                    ReadOnlyMemory<byte>(Encoding.UTF8.GetBytes("{\"some\":\"json data here\"}"))
                )
let eventsList  = List<EventData>()
eventsList.Add(eventData)

Feel free to wrap this in style with a record and functions to make it sexy. Basically we are create one event and adding it to a System.Collections.Generic List. You will likely need to import this.

After that writing events can be done as follows:

try
    let result = client.ConditionalAppendToStreamAsync(streamName, StreamState.Any, eventsList).Result
    match result.Status with
    | ConditionalWriteStatus.Succeeded -> printfn "Events were written!"
    | ConditionalWriteStatus.StreamDeleted -> printfn $"Didn't expect stream to be deleted"
    | ConditionalWriteStatus.VersionMismatch -> printfn $"You should read https://developers.eventstore.com/clients/dotnet/21.2/appending.html#idempotence"
    | _ -> printfn $"unknown outcome! {result.Status}"
with
    | _ as e -> printfn$"{e.Message}"

We are trying to write with ConditionalAppendToStreamAsync, synchronously waiting for the result and matching the status with different outcomes. At the moment we have the versioning to Any, you might want to read about what that means and idempotency in EventStoreDB: https://developers.eventstore.com/clients/dotnet/21.2/appending.html#idempotence

Reading the event written is a bit trickier, you have to use TaskSeq to handle the IAsyncEnumerable returned. So you will first need to import https://github.com/fsprojects/FSharp.Control.TaskSeq

dotnet add package FSharp.Control.TaskSeq --version 0.3.0

You can then do a read backwards, of 1 event (1L means one Long) starting from the end of the stream.

client.ReadStreamAsync(Direction.Backwards,
                       streamName,
                       StreamPosition.End,
                       1L, 
                       true)
    |> TaskSeq.iter (fun event -> printfn$"
                      data: {Encoding.UTF8.GetString(event.OriginalEvent.Data.ToArray())}")
    |> ignore
//probably need to wait here

I hope this has helped you get started with FSharp/F# and EventStoreDB. In this short post we have seen how to write and read events. There are more exciting things that you can do with EventStoreDB, so let me know if you would like to see more posts like this!

Chances are you trying to BenchmarkDotNet to test your Fsharp/F# code. The easiest way to run F# code with BenchmarkDotNet is to build your project in Release mode.

If you really want to debug to the benchmark this is not the right article. This article is about building your FSharp project for Release.

We explore two ways to do this in this article:

1. Using Rider IDE

By default Rider will build the project in Debug mode. So you will need to go to Build->Change Solution Configuration… Then you will need to choose “Release | Any CPU”. Now you can build and run your code and it should work.

2. Using terminal / command line

You will need to add an additional parameter to your dotnet build as follows:

 dotnet build -c Release

The dash c Release tells dotnet to build it for Release. Alternatively you can also use dotnet run as follows:

dotnet run --project YourFSharpProject -c Release

This should be enough to get you going. There are plenty of other articles about how to write BenchmarkDotNet in FSharp so we will not cover it here. Have fun!

1. timeit module

timeit module: This is a built-in Python module that provides a simple way to measure the execution time of small bits of Python code. You can use the timeit.timeit() function to time the execution of a single line of code or multiple lines wrapped in a function.

For example:

import timeit

def some_function():
    # code to be timed
    x = [i**2 for i in range(10000)]

print("Time taken by the function:", timeit.timeit(some_function, number=1000), "seconds")

Sure, here’s a simple example that demonstrates how to use the timeit module to measure the execution time of a piece of code:

python
Copy code
import timeit

def some_function():
# code to be timed
x = [i**2 for i in range(10000)]

print(“Time taken by the function:”, timeit.timeit(some_function, number=1000), “seconds”)
In this example, the timeit.timeit() function is used to measure the execution time of the some_function function. The number parameter is used to specify how many times the function should be run. In this case, it’s set to 1000, so the function will be executed 1000 times and the average time will be returned. The output of the code will look like this:

Time taken by the function: 0.17904642399999998 seconds

You can also use timeit to time a single line of code. For example:

import timeit

print("Time taken by the code:", timeit.timeit('x = [i**2 for i in range(10000)]', number=1000), "seconds")

In this case, the timeit.timeit() function is used to measure the execution time of a single line of code. The number parameter is used to specify how many times the code should be run, and the output will be the average time taken to execute the code.

cProfile module: This is a built-in Python module that provides detailed profiling information about your code, including the time spent in each function and how many times each function was called. You can run this module from the command line to profile your code, or you can use it as a library in your code.

2. cprofile module

This is another built-in Python module that provides detailed profiling information about your code, including the time spent in each function and how many times each function was called. You can run this module from the command line to profile your code, or you can use it as a library in your code.

For example;

import cProfile

def some_function():
    # code to be profiled
    x = [i**2 for i in range(10000)]

cProfile.run("some_function()")

In this example, the cProfile.run() function is used to profile the some_function function. The output of the code will look something like this:

         4 function calls in 0.000 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 profile_example.py:3(some_function)
        1    0.000    0.000    0.000    0.000 profile_example.py:4(<listcomp>)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.exec}

The output shows the number of calls to each function, the total time spent in each function, and the cumulative time spent in each function and its subfunctions. This information can be used to identify performance bottlenecks and optimize your code.

Note that cProfile generates a lot of output, so it’s best to run it on a small piece of code first, and then scale up as needed.

3. line_profiler

This is a third-party package that provides line-by-line profiling information for your code. It can be used to find which lines of your code are taking the most time, and to optimize your code accordingly.

First, you need to install line_profiler using pip (pip install line_profiler) and then, you can use the line_profiler decorator to profile a function. For example:

from line_profiler import LineProfiler

def some_function():
    # code to be profiled
    x = [i**2 for i in range(10000)]

lp = LineProfiler()
lp_wrapper = lp(some_function)
lp_wrapper()
lp.print_stats(

In this example, the LineProfiler class is used to profile the some_function function. The lp_wrapper function is created by passing some_function to the lp instance, and the lp_wrapper function is then executed. Finally, the lp.print_stats() function is used to print the results of the profiling. The output will look something like this:

Timer unit: 1e-06 s

File: <ipython-input-7-1a39a0d7dc68>
Function: some_function at line 5

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     5                                           def some_function():
     6                                               # code to be profiled
     7       1000      278032     278.0     97.3      x = [i**2 for i in range(10000)]

The output shows the number of hits (executions) for each line of code, the total time spent on each line, and the percentage of time spent on each line. This information can be used to identify performance bottlenecks and optimize your code.

4. perf

This is a Linux tool. With perf, you can profile the entire code execution, measure the time taken by individual functions, and determine which functions are taking the most time.

An example:

First, you need to install the Linux perf tool. On Ubuntu, you can do this with:
> sudo apt-get install linux-tools-common

Next, you need to run your Python script with the perf tool, for example:
> perf record -g -F 99 python your_script.py

Finally, you can generate a report of the profile results:
> perf report

In this example, the perf tool is used to profile the your_script.py script. The perf record command is used to start profiling, and the -g option is used to generate call graph information. The -F 99 option is used to specify the sampling frequency (99 Hertz). The perf report command is then used to generate a report of the profile results. The report will show you information such as the function call tree, the number of samples, and the total time spent in each function.

Note that the perf tool is a Linux-specific tool, and is not available on other operating systems. Also, it requires administrative privileges to run.

5. memory_profiler

This is another third-party package that provides detailed information about the memory usage of your code. It can be used to identify memory leaks, monitor memory usage over time, and optimize memory usage in your code.

First, you need to install memory_profiler using “pip install memory_profiler”. Then, you can use the @profile decorator to profile a function. For example:

from memory_profiler import profile

@profile
def some_function():
    # code to be profiled
    x = [i**2 for i in range(10000)]

some_function()

In this example, the @profile decorator is used to profile the some_function function. The some_function function is then executed, and the memory usage of the function is recorded. Finally, the results of the profiling are printed to the console. The output will look something like this:

Line #    Mem usage    Increment   Line Contents
================================================
     4   18.967 MB    0.000 MB   @profile
     5                             def some_function():
     6   18.967 MB    0.000 MB       x = [i**2 for i in range(10000)]

The output shows the memory usage of each line of code, and the incremental change in memory usage between lines. This information can be used to identify memory leaks and optimize your code.

That’s it we hope that this article was helpful. Let us know if you know any other method to benchmark Python code!