val text : string
Multiple items
val string : value:'T -> string

Full name: Microsoft.FSharp.Core.Operators.string

--------------------
type string = System.String

Full name: Microsoft.FSharp.Core.string
val scenePattern : string
val namePattern : string
val parseScenes : sceneAcc:string list list -> characterAcc:string list -> items:string list -> string list list

Full name: index.parseScenes
val sceneAcc : string list list
val characterAcc : string list
val items : string list
type 'T list = List<'T>

Full name: Microsoft.FSharp.Collections.list<_>
val item : string
val rest : string list
active recognizer SceneTitle: string -> Choice<string,string,unit>

Full name: index.( |SceneTitle|Name|Word| )
val title : string
val fullScene : string list
Multiple items
module List

from Microsoft.FSharp.Collections

--------------------
type List<'T> =
  | ( [] )
  | ( :: ) of Head: 'T * Tail: 'T list
  interface IReadOnlyCollection<'T>
  interface IEnumerable
  interface IEnumerable<'T>
  member GetSlice : startIndex:int option * endIndex:int option -> 'T list
  member Head : 'T
  member IsEmpty : bool
  member Item : index:int -> 'T with get
  member Length : int
  member Tail : 'T list
  static member Cons : head:'T * tail:'T list -> 'T list
  static member Empty : 'T list

Full name: Microsoft.FSharp.Collections.List<_>
val rev : list:'T list -> 'T list

Full name: Microsoft.FSharp.Collections.List.rev
active recognizer Name: string -> Choice<string,string,unit>

Full name: index.( |SceneTitle|Name|Word| )
val name : string
active recognizer Word: string -> Choice<string,string,unit>

Full name: index.( |SceneTitle|Name|Word| )
val b : obj

Full name: index.b

The F#orce Awakens

Evelina Gabasova

@evelgab


Warning
Contains some spoilers

script structure

Parsing scripts

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
<pre>
...
    <b> INT. MOS EISLEY SPACEPORT - DOCKING BAY 94 </b>

Chewbacca leads the group into a giant dirt pit that is 
Docking Bay 94. Resting in the middle of the huge hole is a 
large, round, beat-up, pieced-together hunk of junk that 
could only loosely be called a starship.

    <b> LUKE </b>
What a piece of junk.

The tall figure of Han Solo comes down the boarding ramp.
...
</pre>

Parsing scripts

Parsing with active patterns

1: 
2: 
3: 
4: 
5: 
6: 
7: 
8: 
let (|SceneTitle|Name|Word|) (text:string) =
    let scenePattern = "[ 0-9]*(INT.|EXT.)[ A-Z0-9]"
    let namePattern = "^[/A-Z0-9]+[-]*[/A-Z0-9 ]*[-]*[/A-Z0-9 ]+$"
    if Regex.Match(text, scenePattern).Success then
        SceneTitle text
    elif Regex.Match(text, namePattern).Success then
        Name text
    else Word

Parsing scripts with pattern matching

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
let rec parseScenes sceneAcc characterAcc (items: string list) =
   match items with
   | item::rest ->
       match item with
       | SceneTitle title -> 
            // add the finished scene to the scene accumulator
            let fullScene = List.rev characterAcc
            parseScenes (fullScene::sceneAcc) [] rest
       | Name name -> 
            // add character's name to the character accumulator
            parseScenes sceneAcc (name::characterAcc) rest
       | Word -> // do nothing
            parseScenes sceneAcc characterAcc rest
   | [] -> List.rev sceneAcc     

(| Active patterns |)

hide complexity behind readable code

Tion Medon 1

Jupyter notebooks on Azure with F#

notebooks.azure.com/evelina

R2-D2

Chewbacca

Number of common mentions

Number of interactions

Theorem:

There's an API for everything.

Proof:

swapi.co

swapi

swapi-langs

swapi-langs

fsharp-swapi

github.com/evelinag/fsharp-swapi

swapi.co

scenes

+

characters

visualization

Big link to full network
Big link to full network
Big link to full network

the phantom menace

a new hope

the force awakens

Network structure

How do the the movies differ?

Size

Density

Network

Density

Network

Density


\[\begin{align} \text{Density} &= \frac{\text{Existing connections}}{\text{Potential connections}} \\ & \\ &= \frac{\text{Existing connections}}{\frac{1}{2}N(N-1)} \end{align}\]

Density

Clustering coefficient

Network

Clustering coefficient

Clustering

Clustering coefficient

Clustering

Clustering coefficient

Clustering

Clustering coefficient

Clustering

Clustering coefficient

Clustering

Clustering coefficient


\[K_v = \text{Number of neighbours of $v$} \\ E_v = \text{Number of links between neighbours of $v$} \\ \\ \text{Clustering}(v) = \frac{E_v}{\frac{1}{2} K_v (K_v - 1)}\]

Clustering coefficient


\[K_v = \text{Number of neighbours of $v$} \\ E_v = \text{Number of links between neighbours of $v$} \\ \\ \text{Clustering}(\text{network}) = \frac{1}{N} \sum_v \frac{E_v}{\frac{1}{2} K_v (K_v - 1)}\]

Clustering Coefficient

Degree

Network

Degree

Degree

Degree

Degree

Degree


\[\text{Degree}(v) = \text{Number of links }v \leftrightarrow v' \\ v \neq v'\]

Betweenness

Betweenness

Betweenness

Betweenness

Betweenness

Betweenness

Betweenness

Betweenness

Betweenness

Betweenness

Betweenness


\[S_v = \text{Number of shortest paths between $a$ and $b$ through $v$} \\ S = \text{Number of shortest paths between $a$ and $b$} \\ \\ \text{Betweenness}(v)_{ab} = \frac{S_v}{S}\]

Betweenness


\[S_v = \text{Number of shortest paths between $a$ and $b$ through $v$} \\ S = \text{Number of shortest paths between $a$ and $b$} \\ \\ \text{Betweenness}(v) = \sum_{ab} \frac{S_v}{S}\]

R

1: 
2: 
3: 
library(igraph)

b <- betweenness(network)

F#

1: 
2: 
3: 
open RProvider.igraph

let b = R.betweenness(network)

Centrality

Name

Degree

1.

POE

16

2.

FINN

14

3.

HAN

14

4.

CHEWBACCA

12

5.

BB-8

12

Name

Betweenness

1.

POE

97.2

2.

KYLO REN

71.9

3.

REY

38.7

4.

BB-8

29.1

5.

FINN

26.8

Star Wars network in Neo4j

Network analysis in the real world

F# network

@fsharporg

  1. @dsyme
  2. @VisualFSharp
  3. @migueldeicaza
  4. @tomaspetricek
  5. @c4fsharp
November 2014

Parsing scripts in a functional way

Polyglot data science

Type providers

Network science

Having fun with interesting data

F# logo

Learning more

Star Wars resources

Evelina Gabasova

@evelgab

evelinag.com/star-wars-talk

notebooks.azure.com/evelina

github.com/evelinag