go/ferret
1
0
Fork 0
mirror of https://github.com/MontFerret/ferret.git synced 2025-06-25 00:37:26 +02:00
Code Issues Releases Activity

Feature/#9 array functions (#57)

Browse Source 
* #9 Added 'APPEND' function

* #9 Added 'FIRST' function

* #9 Added 'FLATTEN' function

* #9 Added 'INTERSECTION' function

* #9 Added 'LAST' function

* #9 Added 'MINUS' function

* #9 Added 'NTH' function

* #9 Added 'OUTERSECTION' function

* #9 Added 'POP' function

* #9 Added 'POSITION' function

* #9 Added 'PUSH' function

* Fixed nil pointer exception in value parser

* #9 Added 'REMOVE_NTH' function

* #9 Added 'REMOVE_VALUE' function

* #9 Added 'REMOVE_VALUES' function

* #9 Added 'REVERSE' function

* #9 Added 'SHIFT' function

* #9 Added 'SLICE' function

* Removed meme

* #9 Added 'SORTED' function

* #9 Added SORTED_UNIQUE function

* #9 Added 'UNION' function

* #9 Added 'UNION_DISTINCT' function

* #9 Added 'UNIQUE' function

* #9 Added 'UNSHIFT' function

* #9 Made more strict optional arg validation

* #9 Fixed linting errors
This commit is contained in:
Tim Voronov
2018-10-05 21:27:34 -04:00
committed by GitHub
parent 9e947ea9c4
commit ec2d6a659b
53 changed files with 3053 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

375
README.md.orig Normal file
View File

@ -0,0 +1,375 @@
# Ferret
[![Build Status](https://travis-ci.com/MontFerret/ferret.svg?branch=master)](https://travis-ci.com/MontFerret/ferret)
![ferret](https://raw.githubusercontent.com/MontFerret/ferret/master/assets/intro.jpg)
## What is it?
```ferret``` is a web scraping system aiming to simplify data extraction from the web for such things like ui testing, machine learning and analytics.
Having it's own declarative language, ```ferret``` abstracts away technical details and complexity of the underlying technologies, helping to focus on the data itself.
It's extremely portable, extensible and fast.
<<<<<<< HEAD
## Show me some code
=======
## Show me some code
>>>>>>> master
The following example demonstrates the use of dynamic pages.
First of all, we load the main Google Search page, type search criteria into an input box and then click a search button.
The click action triggers a redirect, so we wait till its end.
Once the page gets loaded, we iterate over all elements in search results and assign output to a variable.
The final for loop filters out empty elements that might be because of inaccurate use of selectors.
```aql
LET google = DOCUMENT("https://www.google.com/", true)
INPUT(google, 'input[name="q"]', "ferret")
CLICK(google, 'input[name="btnK"]')
WAIT_NAVIGATION(google)
LET result = (
FOR result IN ELEMENTS(google, '.g')
RETURN {
title: ELEMENT(result, 'h3 > a'),
description: ELEMENT(result, '.st'),
url: ELEMENT(result, 'cite')
}
)
RETURN (
FOR page IN result
FILTER page.title != NONE
RETURN page
)
```
## Features
* Declarative language
* Support of both static and dynamic web pages
* Embeddable
* Extensible
## Motivation
Nowadays data is everything and who owns data - owns the world.
I have worked on multiple data-driven projects where data was an essential part of a system and I realized how cumbersome writing tons of scrapers is.
After some time looking for a tool that would let me to not write a code, but just express what data I need, decided to come up with my own solution.
```ferret``` project is an ambitious initiative trying to bring universal platform for writing scrapers without any hassle.
## Inspiration
FQL (Ferret Query Language) is heavily inspired by [AQL](https://www.arangodb.com/) (ArangoDB Query Language).
But due to the domain specifics, there are some differences in how things work.
## WIP
Be aware, the the project is under heavy development. There is no documentation and some things may change in the final release.
For query syntax, you may go to [ArangoDB web site](https://docs.arangodb.com/3.3/AQL/index.html) and use AQL docs as docs for FQL - since they are identical.
## Installation
### Prerequisites
* Go >=1.6
* GoDep
* GNU Make
* Chrome or Docker (optional)
```sh
go get github.com/MontFerret/ferret
```
You can use your local copy of Google Chrome / Chromium, but for ease of use it's recommended to run it inside a Docker container:
```sh
docker pull alpeware/chrome-headless-trunk
docker run -d -p=0.0.0.0:9222:9222 --name=chrome-headless -v /tmp/chromedata/:/data alpeware/chrome-headless-trunk
```
But if you want to see what's happening during query execution, just start your Chrome with remote debugging port:
```sh
chrome.exe --remote-debugging-port=9222
```
## Quick start
### Browserless mode
If you want to play with ```fql``` and check its syntax, you can run CLI with the following commands:
```
ferret
```
```ferret``` will run in REPL mode.
```shell
Welcome to Ferret REPL
Please use `Ctrl-D` to exit this program.
>%
>LET doc = DOCUMENT('https://news.ycombinator.com/')
>FOR post IN ELEMENTS(doc, '.storylink')
>RETURN post.attributes.href
>%
```
**Note:** symbol ```%``` is used to start and end multi line queries. You also can use heredoc format.
If you want to execute a query stored in a file, just pass a file name:
```
ferret ./docs/examples/static-page.fql
```
```
cat ./docs/examples/static-page.fql | ferret
```
```
ferret < ./docs/examples/static-page.fql
```
### Browser mode
By default, ``ferret`` loads HTML pages via http protocol, because it's faster.
But nowadays, there are more and more websites rendered with JavaScript, and therefore, this 'old school' approach does not really work.
For such cases, you may fetch documents using Chrome or Chromium via Chrome DevTools protocol (aka CDP).
First, you need to make sure that you launched Chrome with ```remote-debugging-port=9222``` flag.
Second, you need to pass the address to ```ferret``` CLI.
```
ferret --cdp http://127.0.0.1:9222
```
**NOTE:** By default, ```ferret``` will try to use this local address as a default one, so it makes sense to explicitly pass the parameter only in case of either different port number or remote address.
Alternatively, you can tell CLI to launch Chrome for you.
```shell
ferret --cdp-launch
```
**NOTE:** Launch command is currently broken on MacOS.
Once ```ferret``` knows how to communicate with Chrome, you can use a function ```DOCUMENT(url, isDynamic)``` with ```true``` boolean value for dynamic pages:
```shell
Welcome to Ferret REPL
Please use `exit` or `Ctrl-D` to exit this program.
>%
>LET doc = DOCUMENT('https://soundcloud.com/charts/top', true)
>WAIT_ELEMENT(doc, '.chartTrack__details', 5000)
>LET tracks = ELEMENTS(doc, '.chartTrack__details')
>FOR track IN tracks
> LET username = ELEMENT(track, '.chartTrack__username')
> LET title = ELEMENT(track, '.chartTrack__title')
> RETURN {
> artist: username.innerText,
> track: title.innerText
> }
>%
```
```shell
Welcome to Ferret REPL
Please use `exit` or `Ctrl-D` to exit this program.
>%
>LET doc = DOCUMENT("https://github.com/", true)
>LET btn = ELEMENT(doc, ".HeaderMenu a")
>CLICK(btn)
>WAIT_NAVIGATION(doc)
>WAIT_ELEMENT(doc, '.IconNav')
>FOR el IN ELEMENTS(doc, '.IconNav a')
> RETURN TRIM(el.innerText)
>%
```
### Embedded mode
```ferret``` is a very modular system and therefore, can be easily be embedded into your Go application.
```go
package main
import (
"context"
"encoding/json"
"fmt"
"github.com/MontFerret/ferret/pkg/compiler"
"os"
)
type Topic struct {
Name string `json:"name"`
Description string `json:"description"`
Url string `json:"url"`
}
func main() {
topics, err := getTopTenTrendingTopics()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
for _, topic := range topics {
fmt.Println(fmt.Sprintf("%s: %s %s", topic.Name, topic.Description, topic.Url))
}
}
func getTopTenTrendingTopics() ([]*Topic, error) {
query := `
LET doc = DOCUMENT("https://github.com/topics")
FOR el IN ELEMENTS(doc, ".py-4.border-bottom")
LIMIT 10
LET url = ELEMENT(el, "a")
LET name = ELEMENT(el, ".f3")
LET desc = ELEMENT(el, ".f5")
RETURN {
name: TRIM(name.innerText),
description: TRIM(desc.innerText),
url: "https://github.com" + url.attributes.href
}
`
comp := compiler.New()
program, err := comp.Compile(query)
if err != nil {
return nil, err
}
out, err := program.Run(context.Background())
if err != nil {
return nil, err
}
res := make([]*Topic, 0, 10)
err = json.Unmarshal(out, &res)
if err != nil {
return nil, err
}
return res, nil
}
```
## Extensibility
That said, ```ferret``` is a very modular system which also allows not only embed it, but extend its standard library.
```go
package main
import (
"context"
"encoding/json"
"fmt"
"github.com/MontFerret/ferret/pkg/compiler"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
"os"
)
func main() {
strs, err := getStrings()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
for _, str := range strs {
fmt.Println(str)
}
}
func getStrings() ([]string, error) {
// function implements is a type of a function that ferret supports as a runtime function
transform := func(ctx context.Context, args ...core.Value) (core.Value, error) {
// it's just a helper function which helps to validate a number of passed args
err := core.ValidateArgs(args, 1)
if err != nil {
// it's recommended to return built-in None type, instead of nil
return values.None, err
}
// this is another helper functions allowing to do type validation
err = core.ValidateType(args[0], core.StringType)
if err != nil {
return values.None, err
}
// cast to built-in string type
str := args[0].(values.String)
return str.Concat(values.NewString("_ferret")).ToUpper(), nil
}
query := `
FOR el IN ["foo", "bar", "qaz"]
// conventionally all functions are registered in upper case
RETURN TRANSFORM(el)
`
comp := compiler.New()
comp.RegisterFunction("transform", transform)
program, err := comp.Compile(query)
if err != nil {
return nil, err
}
out, err := program.Run(context.Background())
if err != nil {
return nil, err
}
res := make([]string, 0, 3)
err = json.Unmarshal(out, &res)
if err != nil {
return nil, err
}
return res, nil
}
```
On top of that, you can completely turn off standard library, by passing the following option:
```go
comp := compiler.New(compiler.WithoutStdlib())
```
And after that, you can easily provide your own implementation of functions from standard library.
If you don't need a particular set of functions from standard library, you can turn off the entire ```stdlib``` and register separate packages from that:
```go
package main
import (
"github.com/MontFerret/ferret/pkg/compiler"
"github.com/MontFerret/ferret/pkg/stdlib/strings"
)
func main() {
comp := compiler.New(compiler.WithoutStdlib())
comp.RegisterFunctions(strings.NewLib())
}
```

80
pkg/runtime/collections/unique.go Normal file
View File

@ -0,0 +1,80 @@
package collections
import (
"github.com/MontFerret/ferret/pkg/runtime/core"
)
type (
UniqueIterator struct {
src Iterator
hashes map[uint64]bool
value core.Value
key core.Value
err error
}
)
func NewUniqueIterator(src Iterator) (*UniqueIterator, error) {
if src == nil {
return nil, core.Error(core.ErrMissedArgument, "source")
}
return &UniqueIterator{
src: src,
hashes: make(map[uint64]bool),
}, nil
}
func (iterator *UniqueIterator) HasNext() bool {
if !iterator.src.HasNext() {
return false
}
iterator.doNext()
if iterator.err != nil {
return false
}
if !core.IsNil(iterator.value) {
return true
}
return false
}
func (iterator *UniqueIterator) Next() (core.Value, core.Value, error) {
return iterator.value, iterator.key, iterator.err
}
func (iterator *UniqueIterator) doNext() {
// reset state
iterator.err = nil
iterator.value = nil
iterator.key = nil
// iterate over source until we find a non-unique item
for iterator.src.HasNext() {
val, key, err := iterator.src.Next()
if err != nil {
iterator.err = err
return
}
h := val.Hash()
_, exists := iterator.hashes[h]
if exists {
continue
}
iterator.hashes[h] = true
iterator.key = key
iterator.value = val
return
}
}

89
pkg/runtime/collections/unique_test.go Normal file
View File

@ -0,0 +1,89 @@
package collections_test
import (
"github.com/MontFerret/ferret/pkg/runtime/collections"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestUniqueIterator(t *testing.T) {
Convey("Should return only unique items", t, func() {
arr := []core.Value{
values.NewInt(1),
values.NewInt(2),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(3),
values.NewInt(5),
values.NewInt(6),
values.NewInt(5),
values.NewInt(6),
}
iter, err := collections.NewUniqueIterator(
collections.NewSliceIterator(arr),
)
So(err, ShouldBeNil)
res, err := collections.ToArray(iter)
So(err, ShouldBeNil)
So(res.String(), ShouldEqual, `[1,2,3,4,5,6]`)
})
Convey("Should return only unique items 2", t, func() {
arr := []core.Value{
values.NewInt(1),
values.NewInt(1),
values.NewInt(1),
values.NewInt(1),
values.NewInt(1),
values.NewInt(1),
}
iter, err := collections.NewUniqueIterator(
collections.NewSliceIterator(arr),
)
So(err, ShouldBeNil)
res, err := collections.ToArray(iter)
So(err, ShouldBeNil)
So(res.String(), ShouldEqual, `[1]`)
})
Convey("Should return only unique items 3", t, func() {
arr := []core.Value{
values.NewString("a"),
values.NewString("b"),
values.NewString("c"),
values.NewString("d"),
values.NewString("e"),
values.NewString("a"),
values.NewString("b"),
values.NewString("f"),
values.NewString("d"),
values.NewString("e"),
values.NewString("f"),
}
iter, err := collections.NewUniqueIterator(
collections.NewSliceIterator(arr),
)
So(err, ShouldBeNil)
res, err := collections.ToArray(iter)
So(err, ShouldBeNil)
So(res.String(), ShouldEqual, `["a","b","c","d","e","f"]`)
})
}

21
pkg/runtime/values/array.go
View File

@ -128,6 +128,10 @@ func (t *Array) ForEach(predicate ArrayPredicate) {
func (t *Array) Get(idx Int) core.Value { func (t *Array) Get(idx Int) core.Value {
l := len(t.value) - 1 l := len(t.value) - 1
if l < 0 {
return None
}
if int(idx) > l { if int(idx) > l {
return None return None
} }
@ -151,8 +155,21 @@ func (t *Array) Push(item core.Value) {
t.value = append(t.value, item) t.value = append(t.value, item)
} }
func (t *Array) Slice(from, to Int) []core.Value { func (t *Array) Slice(from, to Int) *Array {
return t.value[from:to] length := t.Length()
if from >= length {
return NewArray(0)
}
if to > length {
to = length
}
result := new(Array)
result.value = t.value[from:to]
return result
} }
func (t *Array) IndexOf(item core.Value) Int { func (t *Array) IndexOf(item core.Value) Int {

6
pkg/runtime/values/array_test.go
View File

@ -302,12 +302,12 @@ func TestArray(t *testing.T) {
s := arr.Slice(0, 1) s := arr.Slice(0, 1)
So(len(s), ShouldEqual, 1) So(s.Length(), ShouldEqual, 1)
So(s[0].Compare(values.ZeroInt), ShouldEqual, 0) So(s.Get(0).Compare(values.ZeroInt), ShouldEqual, 0)
s2 := arr.Slice(2, arr.Length()) s2 := arr.Slice(2, arr.Length())
So(len(s2), ShouldEqual, arr.Length()-2) So(s2.Length(), ShouldEqual, arr.Length()-2)
}) })
}) })

74
pkg/stdlib/arrays/append.go Normal file
View File

@ -0,0 +1,74 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Appends a new item to an array and returns a new array with a given element.
* If ``uniqueOnly`` is set to true, then will add the item only if it's unique.
* @param arr (Array) - Target array.
* @param item (Value) - Target value to add.
* @returns arr (Array) - New array.
*/
func Append(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, 3)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
arg := args[1]
unique := values.False
if len(args) > 2 {
err = core.ValidateType(args[2], core.BooleanType)
if err != nil {
return values.None, err
}
unique = args[2].(values.Boolean)
}
next := values.NewArray(int(arr.Length()) + 1)
if !unique {
arr.ForEach(func(item core.Value, idx int) bool {
next.Push(item)
return true
})
next.Push(arg)
return next, nil
}
hasDuplicate := false
arr.ForEach(func(item core.Value, idx int) bool {
next.Push(item)
if !hasDuplicate {
hasDuplicate = item.Compare(arg) == 0
}
return true
})
if !hasDuplicate {
next.Push(arg)
}
return next, nil
}

50
pkg/stdlib/arrays/append_test.go Normal file
View File

@ -0,0 +1,50 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/collections"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestAppend(t *testing.T) {
Convey("Should return a copy of an array", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Append(context.Background(), arr, values.NewInt(6))
So(err, ShouldBeNil)
So(out, ShouldNotEqual, arr)
So(out.(collections.Collection).Length(), ShouldBeGreaterThan, arr.Length())
})
Convey("Should ignore non-unique items", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Append(context.Background(), arr, values.NewInt(5), values.True)
So(err, ShouldBeNil)
So(out, ShouldNotEqual, arr)
So(out.(collections.Collection).Length(), ShouldEqual, arr.Length())
out2, err := arrays.Append(context.Background(), arr, values.NewInt(6), values.True)
So(err, ShouldBeNil)
So(out2, ShouldNotEqual, arr)
So(out2.(collections.Collection).Length(), ShouldBeGreaterThan, arr.Length())
})
}

30
pkg/stdlib/arrays/first.go Normal file
View File

@ -0,0 +1,30 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns a first element from a given array.
* @param arr (Array) - Target array.
* @returns element (Value) - First element in a given array.
*/
func First(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 1, 1)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, nil
}
arr := args[0].(*values.Array)
return arr.Get(0), nil
}

35
pkg/stdlib/arrays/first_test.go Normal file
View File

@ -0,0 +1,35 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestFirst(t *testing.T) {
Convey("Should return a first element form a given array", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.First(context.Background(), arr)
So(err, ShouldBeNil)
So(out, ShouldEqual, 1)
})
Convey("Should return NONE if a given array is empty", t, func() {
arr := values.NewArray(0)
out, err := arrays.First(context.Background(), arr)
So(err, ShouldBeNil)
So(out, ShouldEqual, values.None)
})
}

67
pkg/stdlib/arrays/flatten.go Normal file
View File

@ -0,0 +1,67 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Turn an array of arrays into a flat array.
* All array elements in array will be expanded in the result array.
* Non-array elements are added as they are.
* The function will recurse into sub-arrays up to the specified depth.
* Duplicates will not be removed.
* @param arr (Array) - Target array.
* @param depth (Int, optional) - Depth level.
* @returns (Array) - Flat array.
*/
func Flatten(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 1, 2)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
level := 1
if len(args) > 1 {
err = core.ValidateType(args[1], core.IntType)
if err != nil {
return values.None, err
}
level = int(args[1].(values.Int))
}
currentLevel := 0
result := values.NewArray(int(arr.Length()) * 2)
var unwrap func(input *values.Array)
unwrap = func(input *values.Array) {
currentLevel++
input.ForEach(func(value core.Value, idx int) bool {
if value.Type() != core.ArrayType || currentLevel > level {
result.Push(value)
} else {
unwrap(value.(*values.Array))
currentLevel--
}
return true
})
}
unwrap(arr)
return result, nil
}

71
pkg/stdlib/arrays/flatten_test.go Normal file
View File

@ -0,0 +1,71 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestFlatten(t *testing.T) {
Convey("Should flatten an array with depth 1", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewArrayWith(
values.NewInt(3),
values.NewInt(4),
values.NewArrayWith(
values.NewInt(5),
values.NewInt(6),
),
),
values.NewInt(7),
values.NewArrayWith(
values.NewInt(8),
values.NewArrayWith(
values.NewInt(9),
values.NewArrayWith(
values.NewInt(10),
),
),
),
)
out, err := arrays.Flatten(context.Background(), arr)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,3,4,[5,6],7,8,[9,[10]]]")
})
Convey("Should flatten an array with depth more than 1", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewArrayWith(
values.NewInt(3),
values.NewInt(4),
values.NewArrayWith(
values.NewInt(5),
values.NewInt(6),
),
),
values.NewInt(7),
values.NewArrayWith(
values.NewInt(8),
values.NewArrayWith(
values.NewInt(9),
values.NewArrayWith(
values.NewInt(10),
),
),
),
)
out, err := arrays.Flatten(context.Background(), arr, values.NewInt(2))
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,3,4,5,6,7,8,9,[10]]")
})
}

70
pkg/stdlib/arrays/intersection.go Normal file
View File

@ -0,0 +1,70 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Return the intersection of all arrays specified.
* The result is an array of values that occur in all arguments.
* @param arrays (Array, repeated) - An arbitrary number of arrays as multiple arguments (at least 2).
* @returns (Array) - A single array with only the elements, which exist in all provided arrays.
* The element order is random. Duplicates are removed.
*/
func Intersection(_ context.Context, args ...core.Value) (core.Value, error) {
return sections(args, len(args))
}
func sections(args []core.Value, count int) (core.Value, error) {
err := core.ValidateArgs(args, 2, core.MaxArgs)
if err != nil {
return values.None, err
}
intersections := make(map[uint64][]core.Value)
capacity := len(args)
for _, i := range args {
err := core.ValidateType(i, core.ArrayType)
if err != nil {
return values.None, err
}
arr := i.(*values.Array)
arr.ForEach(func(value core.Value, idx int) bool {
h := value.Hash()
bucket, exists := intersections[h]
if !exists {
bucket = make([]core.Value, 0, 5)
}
bucket = append(bucket, value)
intersections[h] = bucket
bucketLen := len(bucket)
if bucketLen > capacity {
capacity = bucketLen
}
return true
})
}
result := values.NewArray(capacity)
required := count
for _, bucket := range intersections {
if len(bucket) == required {
result.Push(bucket[0])
}
}
return result, nil
}

102
pkg/stdlib/arrays/intersection_test.go Normal file
View File

@ -0,0 +1,102 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestIntersection(t *testing.T) {
Convey("Should find intersections between 2 arrays", t, func() {
arr1 := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
)
arr2 := values.NewArrayWith(
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
values.NewInt(7),
values.NewInt(8),
values.NewInt(9),
)
out, err := arrays.Intersection(context.Background(), arr1, arr2)
check := map[int]bool{
4: true,
5: true,
6: true,
}
So(err, ShouldBeNil)
arr := out.(*values.Array)
So(arr.Length(), ShouldEqual, 3)
arr.ForEach(func(value core.Value, idx int) bool {
_, exists := check[int(value.(values.Int))]
So(exists, ShouldBeTrue)
return true
})
})
Convey("Should find intersections between more than 2 arrays", t, func() {
arr1 := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
arr2 := values.NewArrayWith(
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
)
arr3 := values.NewArrayWith(
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
values.NewInt(7),
)
out, err := arrays.Intersection(context.Background(), arr1, arr2, arr3)
check := map[int]bool{
3: true,
4: true,
5: true,
}
So(err, ShouldBeNil)
arr := out.(*values.Array)
So(arr.Length(), ShouldEqual, 3)
arr.ForEach(func(value core.Value, idx int) bool {
_, exists := check[int(value.(values.Int))]
So(exists, ShouldBeTrue)
return true
})
})
}

30
pkg/stdlib/arrays/last.go Normal file
View File

@ -0,0 +1,30 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns the last element of an array.
* @param array (Array) - The target array.
* @returns (Value) - Last element of an array.
*/
func Last(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 1, 1)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, nil
}
arr := args[0].(*values.Array)
return arr.Get(arr.Length() - 1), nil
}

35
pkg/stdlib/arrays/last_test.go Normal file
View File

@ -0,0 +1,35 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestLast(t *testing.T) {
Convey("Should return a last element form a given array", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Last(context.Background(), arr)
So(err, ShouldBeNil)
So(out, ShouldEqual, 5)
})
Convey("Should return NONE if a given array is empty", t, func() {
arr := values.NewArray(0)
out, err := arrays.Last(context.Background(), arr)
So(err, ShouldBeNil)
So(out, ShouldEqual, values.None)
})
}

31
pkg/stdlib/arrays/lib.go Normal file
View File

@ -0,0 +1,31 @@
package arrays
import "github.com/MontFerret/ferret/pkg/runtime/core"
func NewLib() map[string]core.Function {
return map[string]core.Function{
"APPEND": Append,
"FIRST": First,
"FLATTEN": Flatten,
"INTERSECTION": Intersection,
"LAST": Last,
"MINUS": Minus,
"NTH": Nth,
"OUTERSECTION": Outersection,
"POP": Pop,
"POSITION": Position,
"PUSH": Push,
"REMOVE_NTH": RemoveNth,
"REMOVE_VALUE": RemoveValue,
"REMOVE_VALUES": RemoveValues,
"REVERSE": Reverse,
"SHIFT": Shift,
"SLICE": Slice,
"SORTED": Sorted,
"SORTED_UNIQUE": SortedUnique,
"UNION": Union,
"UNION_DISTINCT": UnionDistinct,
"UNIQUE": Unique,
"UNSHIFT": Unshift,
}
}

63
pkg/stdlib/arrays/minus.go Normal file
View File

@ -0,0 +1,63 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Return the difference of all arrays specified.
* @param arrays (Array, repeated) - An arbitrary number of arrays as multiple arguments (at least 2).
* @returns array (Array) - An array of values that occur in the first array, but not in any of the subsequent arrays.
* The order of the result array is undefined and should not be relied on. Duplicates will be removed.
*/
func Minus(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, core.MaxArgs)
if err != nil {
return values.None, err
}
intersections := make(map[uint64]core.Value)
capacity := values.NewInt(0)
for idx, i := range args {
err := core.ValidateType(i, core.ArrayType)
if err != nil {
return values.None, err
}
arr := i.(*values.Array)
arr.ForEach(func(value core.Value, _ int) bool {
h := value.Hash()
// first array, fill out the map
if idx == 0 {
capacity = arr.Length()
intersections[h] = value
return true
}
_, exists := intersections[h]
// if it exists in the first array, remove it
if exists {
delete(intersections, h)
}
return true
})
}
result := values.NewArray(int(capacity))
for _, item := range intersections {
result.Push(item)
}
return result, nil
}

94
pkg/stdlib/arrays/minus_test.go Normal file
View File

@ -0,0 +1,94 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestMinus(t *testing.T) {
Convey("Should find differences between 2 arrays", t, func() {
arr1 := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
)
arr2 := values.NewArrayWith(
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
)
out, err := arrays.Minus(context.Background(), arr1, arr2)
check := map[int]bool{
1: true,
2: true,
}
So(err, ShouldBeNil)
arr := out.(*values.Array)
So(arr.Length(), ShouldEqual, 2)
arr.ForEach(func(value core.Value, idx int) bool {
_, exists := check[int(value.(values.Int))]
So(exists, ShouldBeTrue)
return true
})
})
Convey("Should find differences between more than 2 arrays", t, func() {
arr1 := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
)
arr2 := values.NewArrayWith(
values.NewInt(3),
values.NewInt(9),
values.NewInt(5),
values.NewInt(6),
)
arr3 := values.NewArrayWith(
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
values.NewInt(7),
values.NewInt(8),
)
out, err := arrays.Minus(context.Background(), arr1, arr2, arr3)
check := map[int]bool{
1: true,
2: true,
}
So(err, ShouldBeNil)
arr := out.(*values.Array)
So(arr.Length(), ShouldEqual, 2)
arr.ForEach(func(value core.Value, idx int) bool {
_, exists := check[int(value.(values.Int))]
So(exists, ShouldBeTrue)
return true
})
})
}

40
pkg/stdlib/arrays/nth.go Normal file
View File

@ -0,0 +1,40 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns the element of an array at a given position.
* It is the same as anyArray[position] for positive positions, but does not support negative positions.
* @param array (Array) - An array with elements of arbitrary type.
* @param index (Int) - Position of desired element in array, positions start at 0.
* @returns (Value) - The array element at the given position.
* If position is negative or beyond the upper bound of the array, then NONE will be returned.
*/
func Nth(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, 2)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[1], core.IntType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
idx := args[1].(values.Int)
return arr.Get(idx), nil
}

44
pkg/stdlib/arrays/nth_test.go Normal file
View File

@ -0,0 +1,44 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestNth(t *testing.T) {
Convey("Should return item by index", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Nth(context.Background(), arr, values.NewInt(1))
So(err, ShouldBeNil)
So(out.Compare(values.NewInt(2)), ShouldEqual, 0)
})
Convey("Should return None when no value", t, func() {
arr := values.NewArrayWith()
out, err := arrays.Nth(context.Background(), arr, values.NewInt(1))
So(err, ShouldBeNil)
So(out.Compare(values.None), ShouldEqual, 0)
})
Convey("Should return None when passed negative value", t, func() {
arr := values.NewArrayWith()
out, err := arrays.Nth(context.Background(), arr, values.NewInt(-1))
So(err, ShouldBeNil)
So(out.Compare(values.None), ShouldEqual, 0)
})
}

16
pkg/stdlib/arrays/outersection.go Normal file
View File

@ -0,0 +1,16 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
)
/*
* Return the values that occur only once across all arrays specified.
* @param arrays (Array, repeated) - An arbitrary number of arrays as multiple arguments (at least 2).
* @returns (Array) - A single array with only the elements that exist only once across all provided arrays.
* The element order is random.
*/
func Outersection(_ context.Context, args ...core.Value) (core.Value, error) {
return sections(args, 1)
}

88
pkg/stdlib/arrays/outersection_test.go Normal file
View File

@ -0,0 +1,88 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestOutersection(t *testing.T) {
Convey("Should find intersections between 2 arrays", t, func() {
arr1 := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
)
arr2 := values.NewArrayWith(
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
)
out, err := arrays.Outersection(context.Background(), arr1, arr2)
check := map[int]bool{
1: true,
4: true,
}
So(err, ShouldBeNil)
arr := out.(*values.Array)
So(arr.Length(), ShouldEqual, 2)
arr.ForEach(func(value core.Value, idx int) bool {
_, exists := check[int(value.(values.Int))]
So(exists, ShouldBeTrue)
return true
})
})
Convey("Should find intersections between more than 2 arrays", t, func() {
arr1 := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
)
arr2 := values.NewArrayWith(
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
)
arr3 := values.NewArrayWith(
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Outersection(context.Background(), arr1, arr2, arr3)
check := map[int]bool{
1: true,
5: true,
}
So(err, ShouldBeNil)
arr := out.(*values.Array)
So(arr.Length(), ShouldEqual, 2)
arr.ForEach(func(value core.Value, idx int) bool {
_, exists := check[int(value.(values.Int))]
So(exists, ShouldBeTrue)
return true
})
})
}

44
pkg/stdlib/arrays/pop.go Normal file
View File

@ -0,0 +1,44 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns a new array without last element.
* @param array (Array) - Target array.
* @returns (Array) - Copy of an array without last element.
*/
func Pop(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 1, 1)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
length := int(arr.Length())
result := values.NewArray(length)
lastIdx := length - 1
arr.ForEach(func(value core.Value, idx int) bool {
if idx == lastIdx {
return false
}
result.Push(value)
return true
})
return result, nil
}

35
pkg/stdlib/arrays/pop_test.go Normal file
View File

@ -0,0 +1,35 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestPop(t *testing.T) {
Convey("Should return a copy of an array without last element", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Pop(context.Background(), arr)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,3,4]")
})
Convey("Should return empty array if a given one is empty", t, func() {
arr := values.NewArray(0)
out, err := arrays.Pop(context.Background(), arr)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[]")
})
}

49
pkg/stdlib/arrays/position.go Normal file
View File

@ -0,0 +1,49 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns a value indicating whether an element is contained in array. Optionally returns its position.
* @param array (Array) - The source array.
* @param value (Value) - The target value.
* @param returnIndex (Boolean, optional) - Value which indicates whether to return item's position.
*/
func Position(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, 3)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
el := args[1]
retIdx := false
if len(args) > 2 {
err = core.ValidateType(args[2], core.BooleanType)
if err != nil {
return values.None, err
}
retIdx = args[2].Compare(values.True) == 0
}
position := arr.IndexOf(el)
if !retIdx {
return values.NewBoolean(position > -1), nil
}
return position, nil
}

81
pkg/stdlib/arrays/position_test.go Normal file
View File

@ -0,0 +1,81 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestPosition(t *testing.T) {
Convey("Should return TRUE when a value exists in a given array", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Position(context.Background(), arr, values.NewInt(3))
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "true")
})
Convey("Should return FALSE when a value does not exist in a given array", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Position(context.Background(), arr, values.NewInt(6))
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "false")
})
Convey("Should return index when a value exists in a given array", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Position(
context.Background(),
arr,
values.NewInt(3),
values.NewBoolean(true),
)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "2")
})
Convey("Should return -1 when a value does not exist in a given array", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Position(
context.Background(),
arr,
values.NewInt(6),
values.NewBoolean(true),
)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "-1")
})
}

61
pkg/stdlib/arrays/push.go Normal file
View File

@ -0,0 +1,61 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Create a new array with appended value.
* @param array (Array) - Source array.
* @param value (Value) - Target value.
* @param unique (Boolean, optional) - Value indicating whether to do uniqueness check.
* @returns (Array) - A new array with appended value.
*/
func Push(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, 3)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
value := args[1]
uniq := false
if len(args) > 2 {
err = core.ValidateType(args[2], core.BooleanType)
if err != nil {
return values.None, err
}
uniq = args[2].Compare(values.True) == 0
}
result := values.NewArray(int(arr.Length() + 1))
push := true
arr.ForEach(func(item core.Value, idx int) bool {
if uniq && push {
push = !(item.Compare(value) == 0)
}
result.Push(item)
return true
})
if push {
result.Push(value)
}
return result, nil
}

55
pkg/stdlib/arrays/push_test.go Normal file
View File

@ -0,0 +1,55 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestPush(t *testing.T) {
Convey("Should create a new array with a new element in the end", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Push(context.Background(), arr, values.NewInt(6))
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,3,4,5,6]")
})
Convey("Should not add a new element if not unique when uniqueness check is enabled", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Push(
context.Background(),
arr,
values.NewInt(6),
values.True,
)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,3,4,5,6]")
out2, err := arrays.Push(
context.Background(),
arr,
values.NewInt(6),
values.True,
)
So(out2.String(), ShouldEqual, "[1,2,3,4,5,6]")
})
}

47
pkg/stdlib/arrays/remove_nth.go Normal file
View File

@ -0,0 +1,47 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns a new array without an element by a given position.
* @param array (Array) - Source array.
* @param position (Int) - Target element position.
* @return (Array) - A new array without an element by a given position.
*/
func RemoveNth(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, 2)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[1], core.IntType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
index := int(args[1].(values.Int))
result := values.NewArray(int(arr.Length() - 1))
arr.ForEach(func(value core.Value, idx int) bool {
if idx != index {
result.Push(value)
}
return true
})
return result, nil
}

41
pkg/stdlib/arrays/remove_nth_test.go Normal file
View File

@ -0,0 +1,41 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestRemoveNth(t *testing.T) {
Convey("Should return a copy of an array without an element by its position", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.RemoveNth(context.Background(), arr, values.NewInt(2))
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,4,5]")
})
Convey("Should return a copy of an array with all elements when a position is invalid", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.RemoveNth(context.Background(), arr, values.NewInt(6))
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,3,4,5]")
})
}

64
pkg/stdlib/arrays/remove_value.go Normal file
View File

@ -0,0 +1,64 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns a new array with removed all occurrences of value in a given array.
* Optionally with a limit to the number of removals.
* @param array (Array) - Source array.
* @param value (Value) - Target value.
* @param limit (Int, optional) - A limit to the number of removals.
* @returns (Array) - A new array with removed all occurrences of value in a given array.
*/
func RemoveValue(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, 3)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
value := args[1]
limit := -1
if len(args) > 2 {
err = core.ValidateType(args[2], core.IntType)
if err != nil {
return values.None, err
}
limit = int(args[2].(values.Int))
}
result := values.NewArray(int(arr.Length()))
counter := 0
arr.ForEach(func(item core.Value, idx int) bool {
remove := item.Compare(value) == 0
if remove {
if counter == limit {
result.Push(item)
}
counter++
} else {
result.Push(item)
}
return true
})
return result, nil
}

48
pkg/stdlib/arrays/remove_value_test.go Normal file
View File

@ -0,0 +1,48 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestRemoveValue(t *testing.T) {
Convey("Should return a copy of an array without given element(s)", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(3),
)
out, err := arrays.RemoveValue(context.Background(), arr, values.NewInt(3))
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,4]")
})
Convey("Should return a copy of an array without given element(s) with limit", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(3),
values.NewInt(5),
values.NewInt(3),
)
out, err := arrays.RemoveValue(
context.Background(),
arr,
values.NewInt(3),
values.Int(2),
)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,4,5,3]")
})
}

59
pkg/stdlib/arrays/remove_values.go Normal file
View File

@ -0,0 +1,59 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns a new array with removed all occurrences of values in a given array.
* @param array (Array) - Source array.
* @param values (Array) - Target values.
* @returns (Array) - A new array with removed all occurrences of values in a given array.
*/
func RemoveValues(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, 2)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[1], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
vals := args[1].(*values.Array)
result := values.NewArray(int(arr.Length()))
lookupTable := make(map[uint64]bool)
vals.ForEach(func(value core.Value, idx int) bool {
lookupTable[value.Hash()] = true
return true
})
arr.ForEach(func(value core.Value, idx int) bool {
h := value.Hash()
_, exists := lookupTable[h]
if !exists {
result.Push(value)
}
return true
})
return result, nil
}

35
pkg/stdlib/arrays/remove_values_test.go Normal file
View File

@ -0,0 +1,35 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestRemoveValues(t *testing.T) {
Convey("Should return a copy of an array without given elements", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
)
out, err := arrays.RemoveValues(
context.Background(),
arr,
values.NewArrayWith(
values.NewInt(3),
values.NewInt(5),
values.NewInt(6),
),
)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,4]")
})
}

36
pkg/stdlib/arrays/reverse.go Normal file
View File

@ -0,0 +1,36 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Return a new array with its elements reversed.
* @param array (Array) - Target array.
* @returns (Array) - A new array with its elements reversed.
*/
func Reverse(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 1, 1)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
size := int(arr.Length())
result := values.NewArray(size)
for i := size - 1; i >= 0; i-- {
result.Push(arr.Get(values.NewInt(i)))
}
return result, nil
}

42
pkg/stdlib/arrays/reverse_test.go Normal file
View File

@ -0,0 +1,42 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestReverse(t *testing.T) {
Convey("Should return a copy of an array with reversed elements", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
)
out, err := arrays.Reverse(
context.Background(),
arr,
)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[6,5,4,3,2,1]")
})
Convey("Should return an empty array when there no elements in a source one", t, func() {
arr := values.NewArray(0)
out, err := arrays.Reverse(
context.Background(),
arr,
)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[]")
})
}

41
pkg/stdlib/arrays/shift.go Normal file
View File

@ -0,0 +1,41 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns a new array without the first element.
* @param array (Array) - Target array.
* @returns (Array) - Copy of an array without the first element.
*/
func Shift(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 1, 1)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
length := int(arr.Length())
result := values.NewArray(length)
arr.ForEach(func(value core.Value, idx int) bool {
if idx != 0 {
result.Push(value)
}
return true
})
return result, nil
}

35
pkg/stdlib/arrays/shift_test.go Normal file
View File

@ -0,0 +1,35 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestShift(t *testing.T) {
Convey("Should return a copy of an array without the first element", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Shift(context.Background(), arr)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[2,3,4,5]")
})
Convey("Should return empty array if a given one is empty", t, func() {
arr := values.NewArray(0)
out, err := arrays.Shift(context.Background(), arr)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[]")
})
}

50
pkg/stdlib/arrays/slice.go Normal file
View File

@ -0,0 +1,50 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns a new sliced array.
* @param array (Array) - Source array.
* @param start (Int) - Start position of extraction.
* @param length (Int, optional) - Value indicating how many elements to extract.
* @returns (Array) - Sliced array.
*/
func Slice(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, 3)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[1], core.IntType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
start := args[1].(values.Int)
length := values.NewInt(int(arr.Length()))
if len(args) > 2 {
if args[2].Type() == core.IntType {
arg2 := args[2].(values.Int)
if arg2 > 0 {
length = start + args[2].(values.Int)
}
}
}
return arr.Slice(start, length), nil
}

80
pkg/stdlib/arrays/slice_test.go Normal file
View File

@ -0,0 +1,80 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestSlice(t *testing.T) {
Convey("Should return a sliced array with a given start position ", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
)
out, err := arrays.Slice(context.Background(), arr, values.NewInt(3))
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[4,5,6]")
})
Convey("Should return an empty array when start position is out of bounds", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
)
out, err := arrays.Slice(context.Background(), arr, values.NewInt(6))
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[]")
})
Convey("Should return a sliced array with a given start position and length", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
)
out, err := arrays.Slice(
context.Background(),
arr,
values.NewInt(2),
values.NewInt(2),
)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[3,4]")
})
Convey("Should return an empty array when length is out of bounds", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
values.NewInt(6),
)
out, err := arrays.Slice(context.Background(), arr, values.NewInt(2), values.NewInt(10))
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[3,4,5,6]")
})
}

53
pkg/stdlib/arrays/sorted.go Normal file
View File

@ -0,0 +1,53 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/collections"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Sorts all elements in anyArray.
* The function will use the default comparison order for FQL value types.
* @param array (Array) - Target array.
* @returns (Array) - Sorted array.
*/
func Sorted(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 1, 1)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
if arr.Length() == 0 {
return values.NewArray(0), nil
}
sorter, err := collections.NewSorter(func(first core.Value, second core.Value) (int, error) {
return first.Compare(second), nil
}, collections.SortDirectionAsc)
if err != nil {
return values.None, err
}
iterator, err := collections.NewSortIterator(
collections.NewArrayIterator(arr),
sorter,
)
if err != nil {
return values.None, err
}
return collections.ToArray(iterator)
}

52
pkg/stdlib/arrays/sorted_test.go Normal file
View File

@ -0,0 +1,52 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestSorted(t *testing.T) {
Convey("Should sort numbers", t, func() {
arr := values.NewArrayWith(
values.NewInt(3),
values.NewInt(1),
values.NewInt(6),
values.NewInt(2),
values.NewInt(5),
values.NewInt(4),
)
out, err := arrays.Sorted(context.Background(), arr)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,3,4,5,6]")
})
Convey("Should sort strings", t, func() {
arr := values.NewArrayWith(
values.NewString("b"),
values.NewString("c"),
values.NewString("a"),
values.NewString("d"),
values.NewString("e"),
values.NewString("f"),
)
out, err := arrays.Sorted(context.Background(), arr)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, `["a","b","c","d","e","f"]`)
})
Convey("Should return empty array", t, func() {
arr := values.NewArrayWith()
out, err := arrays.Sorted(context.Background(), arr)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, `[]`)
})
}

60
pkg/stdlib/arrays/sorted_unique.go Normal file
View File

@ -0,0 +1,60 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/collections"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Sorts all elements in anyArray.
* The function will use the default comparison order for FQL value types.
* Additionally, the values in the result array will be made unique
* @param array (Array) - Target array.
* @returns (Array) - Sorted array.
*/
func SortedUnique(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 1, 1)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
if arr.Length() == 0 {
return values.NewArray(0), nil
}
sorter, err := collections.NewSorter(func(first core.Value, second core.Value) (int, error) {
return first.Compare(second), nil
}, collections.SortDirectionAsc)
if err != nil {
return values.None, err
}
uniqIterator, err := collections.NewUniqueIterator(collections.NewArrayIterator(arr))
if err != nil {
return values.None, err
}
iterator, err := collections.NewSortIterator(
uniqIterator,
sorter,
)
if err != nil {
return values.None, err
}
return collections.ToArray(iterator)
}

60
pkg/stdlib/arrays/sorted_unique_test.go Normal file
View File

@ -0,0 +1,60 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestSortedUnique(t *testing.T) {
Convey("Should sort numbers", t, func() {
arr := values.NewArrayWith(
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
values.NewInt(1),
values.NewInt(6),
values.NewInt(2),
values.NewInt(6),
values.NewInt(5),
values.NewInt(1),
values.NewInt(4),
)
out, err := arrays.SortedUnique(context.Background(), arr)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, "[1,2,3,4,5,6]")
})
Convey("Should sort strings", t, func() {
arr := values.NewArrayWith(
values.NewString("e"),
values.NewString("b"),
values.NewString("a"),
values.NewString("c"),
values.NewString("a"),
values.NewString("d"),
values.NewString("f"),
values.NewString("d"),
values.NewString("e"),
values.NewString("f"),
)
out, err := arrays.SortedUnique(context.Background(), arr)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, `["a","b","c","d","e","f"]`)
})
Convey("Should return empty array", t, func() {
arr := values.NewArrayWith()
out, err := arrays.SortedUnique(context.Background(), arr)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, `[]`)
})
}

46
pkg/stdlib/arrays/union.go Normal file
View File

@ -0,0 +1,46 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns the union of all passed arrays.
* @param arrays (Array, repeated) - List of arrays to combine.
* @returns (Array) - All array elements combined in a single array, in any order.
*/
func Union(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, core.MaxArgs)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
firstArrLen := args[0].(*values.Array).Length()
result := values.NewArray(len(args) * int(firstArrLen))
for _, arg := range args {
err := core.ValidateType(arg, core.ArrayType)
if err != nil {
return values.None, err
}
arr := arg.(*values.Array)
arr.ForEach(func(value core.Value, _ int) bool {
result.Push(value)
return true
})
}
return result, nil
}

50
pkg/stdlib/arrays/union_distinct.go Normal file
View File

@ -0,0 +1,50 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
func UnionDistinct(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, core.MaxArgs)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
firstArrLen := args[0].(*values.Array).Length()
result := values.NewArray(len(args) * int(firstArrLen))
hashes := make(map[uint64]bool)
for _, arg := range args {
err := core.ValidateType(arg, core.ArrayType)
if err != nil {
return values.None, err
}
arr := arg.(*values.Array)
arr.ForEach(func(value core.Value, _ int) bool {
h := value.Hash()
_, exists := hashes[h]
if !exists {
hashes[h] = true
result.Push(value)
}
return true
})
}
return result, nil
}

51
pkg/stdlib/arrays/union_distinct_test.go Normal file
View File

@ -0,0 +1,51 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestUnionDistinct(t *testing.T) {
Convey("Should union all arrays with unique values", t, func() {
arr1 := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
)
arr2 := values.NewArrayWith(
values.NewInt(5),
values.NewInt(2),
values.NewInt(6),
values.NewInt(4),
)
arr3 := values.NewArrayWith(
values.NewString("a"),
values.NewString("b"),
values.NewString("c"),
values.NewString("d"),
)
arr4 := values.NewArrayWith(
values.NewString("e"),
values.NewString("b"),
values.NewString("f"),
values.NewString("d"),
)
out, err := arrays.UnionDistinct(
context.Background(),
arr1,
arr2,
arr3,
arr4,
)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, `[1,2,3,4,5,6,"a","b","c","d","e","f"]`)
})
}

53
pkg/stdlib/arrays/union_test.go Normal file
View File

@ -0,0 +1,53 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
/*
* Returns the union of distinct values of all passed arrays.
* @param arrays (Array, repeated) - List of arrays to combine.
* @returns (Array) - All array elements combined in a single array, without duplicates, in any order.
*/
func TestUnion(t *testing.T) {
Convey("Should union all arrays", t, func() {
arr1 := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
)
arr2 := values.NewArrayWith(
values.NewString("a"),
values.NewString("b"),
values.NewString("c"),
values.NewString("d"),
)
arr3 := values.NewArrayWith(
values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
),
values.NewArrayWith(
values.NewInt(3),
values.NewInt(4),
),
)
out, err := arrays.Union(
context.Background(),
arr1,
arr2,
arr3,
)
So(err, ShouldBeNil)
So(out.String(), ShouldEqual, `[1,2,3,4,"a","b","c","d",[1,2],[3,4]]`)
})
}

43
pkg/stdlib/arrays/unique.go Normal file
View File

@ -0,0 +1,43 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/collections"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Returns all unique elements from a given array.
* @param array (Array) - Target array.
* @returns (Array) - New array without duplicates.
*/
func Unique(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 1, 1)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
if arr.Length() == 0 {
return values.NewArray(0), nil
}
iterator, err := collections.NewUniqueIterator(
collections.NewArrayIterator(arr),
)
if err != nil {
return values.None, err
}
return collections.ToArray(iterator)
}

35
pkg/stdlib/arrays/unique_test.go Normal file
View File

@ -0,0 +1,35 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestUnique(t *testing.T) {
Convey("Should return only unique items", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(3),
values.NewInt(5),
values.NewInt(6),
values.NewInt(5),
values.NewInt(6),
)
res, err := arrays.Unique(
context.Background(),
arr,
)
So(err, ShouldBeNil)
So(res.String(), ShouldEqual, `[1,2,3,4,5,6]`)
})
}

80
pkg/stdlib/arrays/unshift.go Normal file
View File

@ -0,0 +1,80 @@
package arrays
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/runtime/values"
)
/*
* Prepends value to a given array.
* @param array (Array) - Target array.
* @param value (Value) - Target value to prepend.
* @param unique (Boolean, optional) - Optional value indicating whether a value must be unique to be prepended.
* Default is false.
* @returns (Array) - New array with prepended value.
*/
func Unshift(_ context.Context, args ...core.Value) (core.Value, error) {
err := core.ValidateArgs(args, 2, 3)
if err != nil {
return values.None, err
}
err = core.ValidateType(args[0], core.ArrayType)
if err != nil {
return values.None, err
}
arr := args[0].(*values.Array)
value := args[1]
uniq := values.False
if len(args) > 2 {
err = core.ValidateType(args[2], core.BooleanType)
if err != nil {
return values.None, err
}
uniq = args[2].(values.Boolean)
}
result := values.NewArray(int(arr.Length() + 1))
if !uniq {
result.Push(value)
arr.ForEach(func(el core.Value, _ int) bool {
result.Push(el)
return true
})
} else {
ok := true
// let's just hope it's unique
// if not, we will terminate the loop and return a copy of an array
result.Push(value)
arr.ForEach(func(el core.Value, idx int) bool {
if el.Compare(value) != 0 {
result.Push(el)
return true
}
// not unique
ok = false
return false
})
if !ok {
// value is not unique, just return a new copy with same elements
return arr.Clone(), nil
}
}
return result, nil
}

59
pkg/stdlib/arrays/unshift_test.go Normal file
View File

@ -0,0 +1,59 @@
package arrays_test
import (
"context"
"github.com/MontFerret/ferret/pkg/runtime/values"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
. "github.com/smartystreets/goconvey/convey"
"testing"
)
func TestUnshift(t *testing.T) {
Convey("Should return a copy of an array", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Unshift(context.Background(), arr, values.NewInt(0))
So(err, ShouldBeNil)
So(out, ShouldNotEqual, arr)
So(out.String(), ShouldEqual, "[0,1,2,3,4,5]")
})
Convey("Should ignore non-unique items", t, func() {
arr := values.NewArrayWith(
values.NewInt(1),
values.NewInt(2),
values.NewInt(3),
values.NewInt(4),
values.NewInt(5),
)
out, err := arrays.Unshift(
context.Background(),
arr,
values.NewInt(0),
values.True,
)
So(err, ShouldBeNil)
So(out, ShouldNotEqual, arr)
So(out.String(), ShouldEqual, "[0,1,2,3,4,5]")
out2, err := arrays.Unshift(
context.Background(),
arr,
values.NewInt(0),
values.True,
)
So(err, ShouldBeNil)
So(out2, ShouldNotEqual, arr)
So(out.String(), ShouldEqual, "[0,1,2,3,4,5]")
})
}

2
pkg/stdlib/lib.go
View File

@ -2,6 +2,7 @@ package stdlib
import ( import (
"github.com/MontFerret/ferret/pkg/runtime/core" "github.com/MontFerret/ferret/pkg/runtime/core"
"github.com/MontFerret/ferret/pkg/stdlib/arrays"
"github.com/MontFerret/ferret/pkg/stdlib/collections" "github.com/MontFerret/ferret/pkg/stdlib/collections"
"github.com/MontFerret/ferret/pkg/stdlib/html" "github.com/MontFerret/ferret/pkg/stdlib/html"
"github.com/MontFerret/ferret/pkg/stdlib/strings" "github.com/MontFerret/ferret/pkg/stdlib/strings"
@ -21,6 +22,7 @@ func NewLib() map[string]core.Function {
add(types.NewLib()) add(types.NewLib())
add(strings.NewLib()) add(strings.NewLib())
add(collections.NewLib()) add(collections.NewLib())
add(arrays.NewLib())
add(html.NewLib()) add(html.NewLib())
add(utils.NewLib()) add(utils.NewLib())