The Mastering the Arcane Art of JavaScript-mancy series are my humble attempt at bringing my love for JavaScript to all other C# developers that haven’t yet discovered how awesome this language and its whole ecosystem are. These articles are excerpts of the super duper awesome JavaScript-Mancy book a compendium of all things JavaScript for C# developers.

What if I told you that JavaScript has LINQ??

In the past few articles we’ve looked into functions in JavaScript, we’ve seen how to work with functions and used different patterns and practices. Today we are going to take that knowledge and apply it into the realm of functional programming.

We’ll start dipping our toes into functional programming by taking a look at how we can use LINQ in JavaScript.

LINQ (Language-Integrated Query) is a .NET query language that lets you work seamlessly with collections of objects (and other data sources). It revolutioned .NET by bringing functional programming concepts to the masses and providing an alternative to the prevalent imperative style programming in C#.

JavaScript offers different ways in which we can achieve that same LINQ-style queries: array prototype functions, ES2015 generators, third-party libraries and ES2016 comprehensions.

Let’s take a look at them!

Functional Programming and JavaScript

Functions are one of the most foundational and common constructs in JavaScript. We use them for all kind of things: to abstract, encapsulate and reuse functionality, as object factories, as object constructors, to simulate data privacy, to create namespaces and modules, we can compose them with other functions, they are everywhere.

The fact that functions are so prevalent in JavaScript, that functions are first-class citizens and that it is very natural to use higher-order functions, make JavaScript a great language to do functional style programming.

One popular style of functional programming is applicative programming when working with collections. In this context applicative programming consists in a function f calling a function g that was given to f originally as an argument. This is easier to understand if we switch f for filter (or Where in LINQ) and g for an arbitrary predicate x => x.status === 'poisoned':

function cureAll(companions) {
  companions.filter(companion => companion.status === 'poisoned').forEach(cure) // cure being defined somewhere else in scope
}
cureAll([Frodo, Sam, Pippin, Merry])
// => You successfully cured Frodo, Sam, Pippin and Merry from poisoning!
// => Good job!

When you get a little bit away from the world of OOP for a moment, away from objects and their characteristic per-piece-of-information micro-language, where you need a specific mini language to speak to or interact with each separate object. When you start moving towards one single data structure, like a collection, and a legion of functions that can operate on this same data structure you can get very interesting results in terms of reusability, flexibility, readability and consistency in your code.

{% blockquote Alan J. Perlis Epigrams On Programming http://pu.inf.uni-tuebingen.de/users/klaeren/epigrams.html %} It is better to have 100 functions operate on one data structure than 10 functions on 10 data structure. {% endblockquote %}

And that’s what LINQ is all about, grabbing a collection of items, performing arbitrary transformations on them to finally get the information in the form and shape that we desire. So let’s see how we can do LINQ in JavaScript.

LINQ in JavaScript

LINQ with Array Prototype Functions

JavaScript array.prototype provides several methods that let you perform operations on array elements in much the same way than LINQ does. For instance:

• map just like Select gives you the possibility to perform transformations in each of the elements of an array and create a new array with these transformed items
• filter just like Where allows you to filter the elements of an array based on a predicate of your choice
• some just like Any returns true when an array contains items that match a given predicate

You can use them like you would in LINQ method syntax:

// LINQ with Array.prototype http://jsfiddle.net/vintharas/es2kg83h/ test it on jsFiddle
// run on Babel.js
function Minion(type, hp) {
  this.type = type
  this.hp = hp || 100
}
Minion.prototype.isAlive = function() {
  return this.hp > 0
}
Minion.prototype.toString = function() {
  return this.type
}

// Array.prototype has a ton of FP methods
// we can make them more LINQ-like
// although they don't support deferred execution
// here I am augmenting Array.prototype so all the arrays within
// your application have these LINQ-named methods
Array.prototype.select = Array.prototype.map
Array.prototype.any = Array.prototype.some
Array.prototype.where = Array.prototype.filter // ES2015
Array.prototype.first = Array.prototype.find // ES2015

function castMany(spell, minions) {
  minions
    .where(function(minion) {
      return minion.isAlive()
    })
    .forEach(spell)
}

function stokeSkin(minion) {
  console.log(
    'all of the sudden the ' +
      minion +
      "'s skin appears hard as rock (+2 constitution)"
  )
}

castMany(stokeSkin, [
  new Minion('orc'),
  new Minion('troll'),
  new Minion('bunny'),
  new Minion('Ghould', /* hp */ -100),
])

/*
=> all of the sudden the orc's skin appears hard as rock (+2 constitution)
=> all of the sudden the troll's skin appears hard as rock (+2 constitution)
=> all of the sudden the bunny's skin appears hard as rock (+2 constitution)
*/

In this example above we augment the Array.prototype object with a series of LINQ aliases to make the existing Array.prototype methods look more like LINQ. By virtue of JavaScript prototypical inheritance all arrays will now expose these new methods. Additionally, it is important to note that the filter and the find methods are part of ES2015 (ES6), so you’ll need babel to run these examples (and if you want to use these features in production today).

And the same example written using arrow functions where I have just switched the anonymous function expressions for arrow functions:

// LINQ with Array.prototype II http://jsfiddle.net/vintharas/rpgu6dqj/ test it on jsFiddle
// run on Babel.js
function Minion(type, hp = 100) {
  this.type = type
  this.hp = hp

  this.isAlive = () => this.hp > 0
  this.toString = () => this.type
}

// since arrow functions get automatically bound to surrounding context
// you cannot use them when setting prototype methods like this:
// Minion.prototype.isAlive = function(){ return this.hp > 0; }
// Minion.prototype.toString = function(){ return this.type; }

Array.prototype.where = Array.prototype.filter // ES2015

function castMany(spell, minions) {
  minions.where(minion => minion.isAlive()).forEach(spell)
}

function stokeSkin(minion) {
  console.log(
    'all of the sudden the ' +
      minion +
      "'s skin appears hard as rock (+2 constitution)"
  )
}

castMany(stokeSkin, [
  new Minion('orc'),
  new Minion('troll'),
  new Minion('bunny'),
  new Minion('Ghould', /* hp */ -100),
])

/*
=> all of the sudden the orc's skin appears hard as rock (+2 constitution)
=> all of the sudden the troll's skin appears hard as rock (+2 constitution)
=> all of the sudden the bunny's skin appears hard as rock (+2 constitution)
*/

And that’s how you can do LINQ in JavaScript… or wait…

LINQ Deferred Execution with ES2015 Generators

There’s something I failed to tell you about in the previous section. The Array.prototype methods are executed as they are called, so every filter, every transformation is executed on every item of an array as soon as the methods are called. That is, there is no concept of deferred execution like we have in LINQ when you use these Array.prototype methods.

Or there was… ES2015 generators bring the yield keyword and deferred evaluation to JavaScript. Let’s see it in action with an example:

// LINQ with ES6 generators http://jsfiddle.net/vintharas/ogkg7huv/ test it on JsFiddle
// run on Babel.js (https://babeljs.io/repl/)
// the * identifies this functions as generators
function* linq(array) {
  for (let item of array) {
    yield item
  }
}

function* where(predicate) {
  for (let item of this) {
    if (predicate(item)) {
      yield item
    }
  }
}

function* select(transform) {
  for (let item of this) {
    yield transform(item)
  }
}

function toList() {
  return [...this]
}

// use a linq-able mixin to wrap linq generators
let linqable = { where, select, toList }
// augment each generator with linq functionality
// to provide the familiar fluent API
Object.assign(/* target */ linq.prototype, linqable)
Object.assign(where.prototype, linqable)
Object.assign(select.prototype, linqable)

var minions = [
  { name: 'orc', hp: 100 },
  { name: 'troll', hp: 40 },
  { name: 'bunny', hp: 1 },
  { name: 'skeleton', hp: 31 },
  { name: 'peon', hp: 10 },
]

var query = linq(minions)
  .where(m => m.hp > 30)
  .select(m => m.name)

console.log("The query hasn't been executed yet")
console.log(`It's value is: ${query}`)
// => It's value is [object Generator]
console.log('At this point it is a generator, awaiting to be iterated')

console.log('We can materialize the query by iterating over the generator')
console.log('Behold! My mightiest Minions!:', query.toList())
// I could also have materialized it by iterating it with a for/of loop, calling the
// query.next() method explicitly, etc
// for (let item of query)
//  console.log(item)

In this example we created several generators that allow us to iterate over an array in the manner of our choosing and with deferred evaluation. In order to get the fluent API experience, we also extended the prototype of each of these generators using a mixin and the new Object.assign native ES2015(ES6) method that let us extend any object with new properties and methods (like $.extend or _.extend).

But this feels a little bit unwieldy, doesn’t it. To make it more usable, we could add these new LINQ capabilities to the array.prototype itself:

// LINQ with ES6 generators and array.prototype http://jsfiddle.net/vintharas/716k6tx2/ test it on JsFiddle
// in addition to the code above
// adding LINQ generator-based functionality to arrays
Array.prototype.toLinqable = function() {
  return linq(this)
}
Object.assign(Array.prototype, linqable)

var minions = [
  { name: 'orc', hp: 100 },
  { name: 'troll', hp: 40 },
  { name: 'bunny', hp: 1 },
  { name: 'skeleton', hp: 31 },
  { name: 'peon', hp: 10 },
]

var query = minions
  .toLinqable()
  .where(m => m.hp > 30)
  .select(m => m.name)

console.log("The query hasn't been executed yet")
console.log(`It's value is: ${query}`)
// => It's value is [object Generator]
console.log('At this point it is a generator, awaiting to be iterated')

console.log('We can materialize the query by iterating over the generator')
console.log('Behold! My mightiest Minions!:', query.toList())

// you can now use LINQ with any array
console.log(
  [1, 2, 3, 4, 5, 6, 7]
    .toLinqable()
    .where(n => n > 5)
    .toList()
)
// => 6,7

And that is much nicer indeed. Anyhow, you’ll probably don’t want to write and maintain your own version of LINQ and instead rely on one of the very many popular libraries within the thriving JavaScript ecosystem. Let’s look at those next.

Filling in the Gaps with JavaScript Libraries

One of the most awesome things of the JavaScript ecosystem is the huge amount of open source projects there are out there. Whatever the language or the browser lacks, the community makes up for. As a result, you have libraries for anything that you can image, and FP programming with collections is not an exception.

Without a doubt, the two most popular libraries for writing LINQ-like expressions are underscore and lodash. They provide very similar APIs and you could consider lo-dash a superset of underscore. You can read this great article by @ben336 where he compares both libraries.

Let’s take a look at how the previous example would look when written in underscore:

// LINQ with underscore http://jsfiddle.net/vintharas/4zvo06v0/ test it on jsFiddle
// let's rewrite the previous examples with underscore instead xD
// the first example went something like this...

function Minion(type, hp) {
  this.type = type
  this.hp = hp || 100
}
Minion.prototype.isAlive = function() {
  return this.hp > 0
}
Minion.prototype.toString = function() {
  return this.type
}

function castMany(spell, minions) {
  // underscore chain lets you chain
  // method calls together
  _.chain(minions)
    .filter(function(minion) {
      return minion.isAlive()
    })
    .each(spell)
    .value()
}

function stokeSkin(minion) {
  console.log(
    'all of the sudden the ' +
      minion +
      "'s skin appears hard as rock (+2 constitution)"
  )
}

castMany(stokeSkin, [
  new Minion('orc'),
  new Minion('troll'),
  new Minion('bunny'),
  new Minion('Ghould', -100),
])

// and the second example...
var minions = [
  new Minion(/*name:*/ 'orc', /*hp:*/ 100),
  new Minion('troll', 40),
  new Minion('bunny', 1),
  new Minion('skeleton', 31),
  new Minion('peon', 10),
]

var healthyMinions = _.chain(minions)
  .filter(function(m) {
    // The chain doesn't get evaluated until you call value
    // so that's another way to get deferred evaluation
    console.log('calling this function with: ' + m)
    return m.hp > 30
  })
  .map(function(m) {
    return m.type
  })

console.log("the chain hasn't been evaluated yet")
console.log("let's look at what exactly healthyMinions is: ")
console.log(healthyMinions)
// => w {_wrapped: Array[3], _chain: true}
console.log('hmm some sort of underscore wrapper object :)')
console.log(healthyMinions.value())
// => [orc, troll, skeleton]

Since lodash is a superset of underscore, if you switch libraries in the previous example it will just work. One cool thing that you can do with lodash that you can’t with underscore is implicit chaining (by losing deferred evaluation):

// in adition to
var mightiestMinions = _.chain(minions)
  .filter(function(m) {
    return m.hp > 30
  })
  .map(function(m) {
    return m.type
  })
  .value()
console.log(mightiestMinions)
// =>  [orc, troll, skeleton]

// you can do this
var mightiestMinions = _(minions)
  .filter(function(m) {
    return m.hp > 30
  })
  .map(function(m) {
    return m.type
  })
console.log(mightiestMinions)
// =>  [orc, troll, skeleton]

// if you were to attempt that with underscore
// the filter would directly return a native array
// instead of an underscore object

There is also a direct LINQ port to JavaScript linq.js, that will give you the most LINQ-like experience that you can find in JavaScript. And finally, we’ve got wu.js, a library that takes advantage of ES2015 (ES6) iterators and generators. Using iterators and generators gives you true, native deferred evaluation that automatically integrates with new ES2015 (ES6) features like the for/of loop and the ... spread operator.

LINQ Query Syntax with ES2016 Comprenhensions

A new super-duper-useful feature that will come with ES2016(ES7) are comprehensions. They provide a query-like syntax to operating with arrays (or any iterable data structure). If you are familiar with Python, Ruby or CoffeeScript you may have encountered comprehensions before. They look like this:

// array comprehensions http://jsfiddle.net/vintharas/86uefx9c/ test it on jsFiddle
// Run on babel.js
// with the experimental flag on

var healthyMinions = [for (m of minions)
                       if (m.isAlive && m.hp > 50)
                         m];

console.log(healthyMinions);
console.log("Behold! These are my healthy minions!", healthyMinions);
// => Behold! These are my healthy minions! [{"type": "bunny", "hp": 100}]

And another example working on nested collections:

// array comprehensions http://jsfiddle.net/vintharas/86uefx9c/ test it on jsFiddle
var fortresses = [
  {name: "Castle Doom",
   minions: [{name: "troll", type: "elite", hp: 1000},
             {name: "orc", type: "normal", hp: 50},
             {name: "warg", type: "normal", hp: 100}]
  },
  {name: "Fortress Of Solitude",
   minions: [{name: "superman", type: "hero", hp: 99999}]
  },
  {name: "GreySkull",
   minions: [{name: "He-Man", type: "elite", hp: 5000}]
  },
  {name: "Cave",
   minions: [{name: "TheGrue", type: "elite", hp: 10000}]
  }
];

// you can also traverse nested collections
var eliteMinions = [for (f of fortresses)
                      for (m of f.minions)
                        if (m.type == "elite")
                          m];
console.log("Behold! My minions!: ", eliteMinions);
// => Behold! My minions!: [{"name": "troll", "type": "elite", ...

You can start experimenting with this new JavaScript feature today using Babel in experimental mode, but the specification is in a very early stage (level 0 strawman proposal) so the syntax and behavior are subject to change as the proposal goes through the standardization process.

Additionally, ES2016 (ES7) will also bring generator comprehensions that will look like array comprehensions but between parenthesis (<my comprehension here>) instead of square brackets []. They will behave just like array comprehensions but with deferred evaluation.

Concluding

And that’s it! This felt like a long post (at least to me while I was writing it). I really hope you’ve enjoyed this one. We covered a lot of different ways in which you can do LINQ in javascript, in summary:

• We saw how you can use the Array.prototype methods for a native implementation of LINQ-like functionality. One the plus side, since they are native you can assume that they’ll be faster that non-native implementations and they don’t require you to add any additional library. On the minus side, there are not so very many methods, some of them are only available from ES5, and others from ES6, so there are various degrees of support depending on the browsers that you are targeting. They are also executed as they are called, so you don’t get deferred evaluation.
• We saw how you can use ES6 generators to add deferred evaluation into the mix. Nice to know how generators work but you’ll probably don’t want to roll and maintain your own implementation of LINQ, instead you’ll probably…
• Use one of the many libraries within the JavaScript ecosystem. Underscore and lodash are the most popular, but there’s even a LINQ port called linq.js and a new library called wu.js that has support for ES6 iterators and generators. On the plus side any of these libraries provides infinitely more functionality than the Array.prototype methods, they behave consistently across many browsers and will usually provide pollyfills and fall back to native APIs if they exist. On the minus side they are an additional dependency that you need to manage, and a new library to learn how to use.
• Finally, we took a look at some cool things that are coming in the next version of JavaScript (ES2016/ES7) which are array and generator comprehensions. Comprehensions let you easily perform operations on collections with a very readable query syntax.

In the next chapter of this series I will digress a little bit from functional programming and take a look at ES2015 (ES6) iterators and generators, to later come back to functional programming with function composition and building abstractions on top of abstractions on top of abstractions :).

Want to Learn Some More?

Here you can find more information related to this article:

• MDN on the Array.prototype object
• MDN on Iterators and Generators in JavaScript
• Kyle Simpson’s Great Series on Generators
• 2ality on ES6 Generators in Depth
• MDN on Comprehensions
• ECMAScript proposal for array comprehensions

Interested in Learning More JavaScript? Buy the Book!

Are you a C# Developer interested in learning JavaScript? Then take a look at the JavaScript-mancy book, a complete compendium of JavaScript for C# developers.

Have a great week ahead!! :)

More Articles In This Series

• Chapter 0: The Basic Ingredients of JavaScriptMancy
• Chapter 1: The Many a One JavaScript Quirks
• Functions
  • Chapter 2: The Basics of JavaScript Functions
  • Chapter 3: Useful function patterns – Default Arguments
  • Chapter 4: More useful function patterns – Multiple Arguments
  • Chapter 5: Even More useful function patterns – Function Overloading
  • Chapter 6: Functions in ES2015
• Object Oriented Programming
  • On Summoning Servants and Critters, Or The Basics of Objects
• Basics
  • A Guide to Strings, Finding The Right Words and Proper Spell Intonation
• ES6
  • ES6 Destructuring
  • ES6 Arrow Functions
  • ES6 Spread Operator
• OOP
  • Introduction to OOP in JavaScript for C# Developers
  • Summoning Fundamentals: Introduction to OOP In JavaScript – Encapsulation
  • Summoning Fundamentals: Introduction to OOP In JavaScript – Inheritance
  • Summoning Fundamentals: Introduction to OOP In JavaScript – Polymorphism
  • White Tower Summoning: Mimicking C# Classical Inheritance in Javascript
  • White Tower Summoning Enhanced: The Marvels of ES6 Classes
  • Black Tower Summoning: Object Composition with Mixins
  • Safer JavaScript Composition With Traits and Traits.js
  • JavaScript Ultra Flexible Object Oriented Programming with Stamps
  • Object Oriented JavaScript for C# Programmers
• Tooling
  • Elevating Your Front-End Workflow With Yeoman
  • Automate Your Front-end Workflow with Gulp – Getting Started
• Data Structures
  • Arrays
  • Maps
  • Sets
  • Enumerables, Iterables, Iterators and Generators
• Functional Programming
  • Functional Programming in JavaScript
  • Using LINQ in JavaScript
• And more stuff to be shaped out:
  • Functional Programming with JavaScript
  • Modules
  • Asynchronous Programming
  • The JavaScript Ecosystem