wpetit
/
super-graph
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Fix bugs and document the 'where' clause

This commit is contained in:
Vikram Rangnekar 2019-03-27 02:58:19 -04:00
parent b0083d99ef
commit 59b6a4a368
5 changed files with 368 additions and 187 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

60
README.md
View File

@ -2,13 +2,13 @@
## Instant GraphQL API for Rails. Zero code.
Get an instant high-performance GraphQL API for your Rails apps in seconds. Super Graph will auto learn your database structure and relationships. Built in support for Rails authentication and for JWT tokens.
Get an high-performance GraphQL API for your Rails app in seconds. Super Graph will auto learn your database structure and relationships. Built in support for Rails authentication and JWT tokens.
## Back story and motivation
I have a Rails app that gets a bit of traffic. Having planned to improve the UI using React or Vue I found that my current APIs didn't have the data I needed. I was too lazy to build new controllers. My controllers were esentially wrappers around database queries and I didn't enjoy having to figure out new REST APIs with paths, names and methods to fetch all this new data.
I have a Rails app that gets a bit of traffic. Having planned to improve the UI using React or Vue I found that my current APIs didn't have the data I needed and I was too lazy to build new endpoints. My controllers were esentially wrappers around database queries and I didn't enjoy having to figure out new REST APIs with paths, names and methods to fetch all this new data or write the active record code needed.
I always liked GraphQL and how simplifies things for web devs. On the backend however GraphQL seemed overly complex as it still required me to write a lot of the same database query code. I wanted a GraphQL server that just worked the second you deployed it without having to write a line of code.
I always liked GraphQL and how simple it makes things for web devs. On the backend however GraphQL seemed overly complex as it still required me to write a lot of the same database query code. I wanted a GraphQL server that just worked the second you deployed it without having to write a line of code.
And so after a lot of coffee and some avocado toasts we now have Super Graph, an instant GraphQL API that is high performance and quick to deploy. One service to rule all your database querying needs.
@ -17,9 +17,9 @@ And so after a lot of coffee and some avocado toasts we now have Super Graph, an
- Belongs-To, One-To-Many and Many-To-Many table relationships
- Devise, Warden encrypted and signed session cookies
- Redis, Memcache and Cookie session stores
- Generates highly optimized Postgres SQL quries
- Generates highly optimized Postgres SQL queries
- Customize through a simple config file
- High performance GoLang codebase
- High performance GO codebase
- Tiny docker image and low memory requirements
### GraphQL (GQL)
@ -50,7 +50,6 @@ query {
The above GQL query returns the JSON result below. It handles all
kinds of complexity without you writing a line of code. For example there is a while greater than `gt` and a limit clause on a child field. And the `avatar` field is renamed to `picture`. The `password` field is blocked and not returned. Finally the relationship between the `users` table and the `products` table is auto discovered and used.
#### JSON Result
```json
@ -76,18 +75,18 @@ kinds of complexity without you writing a line of code. For example there is a w
}
}
```
## Try it out
Please be patient on the first run Go has to download packages and this
can be a little slow.
```console
$ docker-compose run web rake db:create db:migrate db:seed
$ docker-compose up
$ docker-compose -f docker-compose.image.yml up
$ open http://localhost:8080
```
The above command will download the latest docker image for Super Graph and use it to run an example that includes a Postgres DB and a simple Rails ecommerce store app. If you want to build and run Super Graph from code then use the following command instead `docker-compose up`
#### How to try with an authenticated user
In development mode you can use the `X-User-ID: 4` header to set a user id so you don't have to worries about cookies etc. This can be set using the *HTTP Headers* tab at the bottom of the web UI you'll see when you visit the above link. You can also directly run quries from the commandline like shown below.
#### Querying the GQL endpoint
@ -99,11 +98,9 @@ curl 'http://localhost:8080/api/v1/graphql' \
--data-binary '{"query":"{ products { name price users { email }}}"}'
```
## How to GQL
## How to GraphQL
GQL is a simple query language that is fast replacing REST APIs. GQL is great
since it allows web developers to fetch the exact data that they need without
depending on changes to backend code.
GraphQL / GQL is a simple query syntax that is fast replacing REST APIs. GQL is great since it allows web developers to fetch the exact data that they need without depending on changes to backend code. Also if you squint hard enough it looks a little bit like JSON :)
The below query will fetch a `users` name, email and avatar image renamed as picture. If you also need the users `id` then just add it to the query.
@ -119,6 +116,39 @@ query {
Super Graph support complex quries where you can add filters, ordering, offsets and limits on the query.
#### Logical Operators
Name | Example | Explained |
--- | --- | --- |
and | price : { and : { gt: 10.5, lt: 20 } | price > 10.5 AND price < 20
or | or : { price : { greater_than : 20 }, quantity: { gt : 0 } } | price >= 20 OR quantity > 0
not | not: { or : { quantity : { eq: 0 }, price : { eq: 0 } } } | NOT (quantity = 0 OR price = 0)
#### Other conditions
Name | Example | Explained |
--- | --- | --- |
eq, equals | id : { eq: 100 } | id = 100
neq, not_equals | id: { not_equals: 100 } | id != 100
gt, greater_than | id: { gt: 100 } | id > 100
lt, lesser_than | id: { gt: 100 } | id < 100
gte, greater_or_equals | id: { gte: 100 } | id >= 100
lte, lesser_or_equals | id: { lesser_or_equals: 100 } | id <= 100
in | status: { in: [ "A", "B", "C" ] } | status IN ('A', 'B', 'C)
nin, not_in | status: { in: [ "A", "B", "C" ] } | status IN ('A', 'B', 'C)
like | name: { like "phil%" } | Names starting with 'phil'
nlike, not_like | name: { nlike "v%m" } | Not names starting with 'v' and ending with 'm'
ilike | name: { ilike "%wOn" } | Names ending with 'won' case-insensitive
nilike, not_ilike | name: { nilike "%wOn" } | Not names ending with 'won' case-insensitive
similar | name: { similar: "%(b\|d)%" } | [Similar Docs](https://www.postgresql.org/docs/9/functions-matching.html#FUNCTIONS-SIMILARTO-REGEXP)
nsimilar, not_similar | name: { nsimilar: "%(b\|d)%" } | [Not Similar Docs](https://www.postgresql.org/docs/9/functions-matching.html#FUNCTIONS-SIMILARTO-REGEXP)
has_key | column: { has_key: 'b' } | Does JSON column contain this key
has_key_any | column: { has_key_any: [ a, b ] } | Does JSON column contain any of these keys
has_key_all | column: [ a, b ] | Does JSON column contain all of this keys
contains | column: { contains: [1, 2, 4] } | Is this array/json column a subset of value
contained_in | column: { contains: "{'a':1, 'b':2}" } | Is this array/json column a subset of these value
is_null | column: { is_null: true } | Is column value null or not
```javascript
query {
products(

39
psql/psql.go
View File

@ -4,6 +4,7 @@ import (
"errors"
"fmt"
"io"
"strings"
"github.com/dosco/super-graph/qcode"
"github.com/dosco/super-graph/util"
@ -385,19 +386,17 @@ func (v *selectBlock) renderWhere(w io.Writer) error {
}
case *qcode.Exp:
switch val.Op {
case qcode.OpAnd:
st.Push(val.Children[1])
st.Push(qcode.OpAnd)
st.Push(val.Children[0])
continue
case qcode.OpOr:
st.Push(val.Children[1])
st.Push(qcode.OpOr)
st.Push(val.Children[0])
case qcode.OpAnd, qcode.OpOr:
for i := len(val.Children) - 1; i >= 0; i-- {
st.Push(val.Children[i])
if i > 0 {
st.Push(val.Op)
}
}
continue
case qcode.OpNot:
st.Push(qcode.OpNot)
st.Push(val.Children[0])
st.Push(qcode.OpNot)
continue
}
@ -406,6 +405,7 @@ func (v *selectBlock) renderWhere(w io.Writer) error {
} else {
fmt.Fprintf(w, `(("%s"."%s") `, v.sel.Table, val.Col)
}
valExists := true
switch val.Op {
case qcode.OpEquals:
@ -437,20 +437,33 @@ func (v *selectBlock) renderWhere(w io.Writer) error {
case qcode.OpNotSimilar:
io.WriteString(w, `NOT SIMILAR TO`)
case qcode.OpContains:
io.WriteString(w, `CONTAINS`)
io.WriteString(w, `@>`)
case qcode.OpContainedIn:
io.WriteString(w, `CONTAINED IN`)
io.WriteString(w, `<@`)
case qcode.OpHasKey:
io.WriteString(w, `HAS KEY`)
io.WriteString(w, `?`)
case qcode.OpHasKeyAny:
io.WriteString(w, `?|`)
case qcode.OpHasKeyAll:
io.WriteString(w, `?&`)
case qcode.OpIsNull:
if strings.EqualFold(val.Val, "true") {
io.WriteString(w, `IS NULL`)
} else {
io.WriteString(w, `IS NOT NULL`)
}
valExists = false
default:
return fmt.Errorf("[Where] unexpected op code %d", val.Op)
}
if valExists {
if val.Type == qcode.ValList {
renderList(w, val)
} else {
renderVal(w, val, v.vars)
}
}
io.WriteString(w, `)`)

82
psql/psql_test.go
View File

@ -110,7 +110,7 @@ func compileGQLToPSQL(gql string) (string, error) {
return sqlStmt.String(), nil
}
func TestCompileGQLWithArgs(t *testing.T) {
func TestCompileGQLWithComplexArgs(t *testing.T) {
gql := `query {
products(
# returns only 30 items
@ -145,6 +145,86 @@ func TestCompileGQLWithArgs(t *testing.T) {
}
}
func TestCompileGQLWithWhereMultiOr(t *testing.T) {
gql := `query {
products(
where: {
or: {
not: { id: { is_null: true } },
price: { gt: 10 },
price: { lt: 20 }
} }
) {
id
name
price
}
}`
sql := `SELECT json_object_agg('products', products) FROM (SELECT coalesce(json_agg("products"), '[]') AS "products" FROM (SELECT row_to_json((SELECT "sel_0" FROM (SELECT "products_0"."id" AS "id", "products_0"."name" AS "name", "products_0"."price" AS "price") AS "sel_0")) AS "products" FROM (SELECT "products"."id", "products"."name", "products"."price" FROM "products" WHERE ((("products"."price") < (20)) OR (("products"."price") > (10)) OR NOT (("products"."id") IS NULL)) LIMIT ('20') :: integer) AS "products_0" LIMIT ('20') :: integer) AS "products_0") AS "done_1337";`
resSQL, err := compileGQLToPSQL(gql)
if err != nil {
t.Fatal(err)
}
if resSQL != sql {
t.Fatal(errNotExpected)
}
}
func TestCompileGQLWithWhereIsNull(t *testing.T) {
gql := `query {
products(
where: {
and: {
not: { id: { is_null: true } },
price: { gt: 10 }
}}) {
id
name
price
}
}`
sql := `SELECT json_object_agg('products', products) FROM (SELECT coalesce(json_agg("products"), '[]') AS "products" FROM (SELECT row_to_json((SELECT "sel_0" FROM (SELECT "products_0"."id" AS "id", "products_0"."name" AS "name", "products_0"."price" AS "price") AS "sel_0")) AS "products" FROM (SELECT "products"."id", "products"."name", "products"."price" FROM "products" WHERE ((("products"."price") > (10)) AND NOT (("products"."id") IS NULL)) LIMIT ('20') :: integer) AS "products_0" LIMIT ('20') :: integer) AS "products_0") AS "done_1337";`
resSQL, err := compileGQLToPSQL(gql)
if err != nil {
t.Fatal(err)
}
if resSQL != sql {
t.Fatal(errNotExpected)
}
}
func TestCompileGQLWithWhereAndList(t *testing.T) {
gql := `query {
products(
where: {
and: [
{ not: { id: { is_null: true } } },
{ price: { gt: 10 } },
] } ) {
id
name
price
}
}`
sql := `SELECT json_object_agg('products', products) FROM (SELECT coalesce(json_agg("products"), '[]') AS "products" FROM (SELECT row_to_json((SELECT "sel_0" FROM (SELECT "products_0"."id" AS "id", "products_0"."name" AS "name", "products_0"."price" AS "price") AS "sel_0")) AS "products" FROM (SELECT "products"."id", "products"."name", "products"."price" FROM "products" WHERE ((("products"."price") > (10)) AND NOT (("products"."id") IS NULL)) LIMIT ('20') :: integer) AS "products_0" LIMIT ('20') :: integer) AS "products_0") AS "done_1337";`
resSQL, err := compileGQLToPSQL(gql)
if err != nil {
t.Fatal(err)
}
if resSQL != sql {
t.Fatal(errNotExpected)
}
}
func TestCompileGQLOneToMany(t *testing.T) {
gql := `query {
users {

37
qcode/parse.go
View File

@ -332,14 +332,11 @@ func (p *Parser) parseArgs() ([]*Arg, error) {
return args, nil
}
func (p *Parser) parseList(parent *Node) ([]*Node, error) {
func (p *Parser) parseList() (*Node, error) {
parent := &Node{}
var nodes []*Node
var ty parserType
if parent == nil {
return nil, errors.New("list needs a parent")
}
for {
if p.peek(itemListClose) {
p.ignore()
@ -356,6 +353,7 @@ func (p *Parser) parseList(parent *Node) ([]*Node, error) {
return nil, errors.New("All values in a list must be of the same type")
}
}
node.Parent = parent
nodes = append(nodes, node)
}
if len(nodes) == 0 {
@ -365,16 +363,13 @@ func (p *Parser) parseList(parent *Node) ([]*Node, error) {
parent.Type = nodeList
parent.Children = nodes
return nodes, nil
return parent, nil
}
func (p *Parser) parseObj(parent *Node) ([]*Node, error) {
func (p *Parser) parseObj() (*Node, error) {
parent := &Node{}
var nodes []*Node
if parent == nil {
return nil, errors.New("object needs a parent")
}
for {
if p.peek(itemObjClose) {
p.ignore()
@ -403,33 +398,22 @@ func (p *Parser) parseObj(parent *Node) ([]*Node, error) {
parent.Type = nodeObj
parent.Children = nodes
return nodes, nil
return parent, nil
}
func (p *Parser) parseValue() (*Node, error) {
node := &Node{}
var done bool
var err error
if p.peek(itemListOpen) {
p.ignore()
node.Children, err = p.parseList(node)
done = true
return p.parseList()
}
if p.peek(itemObjOpen) {
p.ignore()
node.Children, err = p.parseObj(node)
done = true
return p.parseObj()
}
if err != nil {
return nil, err
}
if !done {
item := p.next()
node := &Node{}
switch item.typ {
case itemIntVal:
@ -448,7 +432,6 @@ func (p *Parser) parseValue() (*Node, error) {
return nil, fmt.Errorf("expecting a number, string, object, list or variable as an argument value (not %s)", p.next().val)
}
node.Val = item.val
}
return node, nil
}

273
qcode/qcode.go
View File

@ -86,6 +86,60 @@ const (
OpIsNull
)
func (t ExpOp) String() string {
var v string
switch t {
case OpAnd:
v = "op-and"
case OpOr:
v = "op-or"
case OpNot:
v = "op-not"
case OpEquals:
v = "op-equals"
case OpNotEquals:
v = "op-not-equals"
case OpGreaterOrEquals:
v = "op-greater-or-equals"
case OpLesserOrEquals:
v = "op-lesser-or-equals"
case OpGreaterThan:
v = "op-greater-than"
case OpLesserThan:
v = "op-lesser-than"
case OpIn:
v = "op-in"
case OpNotIn:
v = "op-not-in"
case OpLike:
v = "op-like"
case OpNotLike:
v = "op-not-like"
case OpILike:
v = "op-i-like"
case OpNotILike:
v = "op-not-i-like"
case OpSimilar:
v = "op-similar"
case OpNotSimilar:
v = "op-not-similar"
case OpContains:
v = "op-contains"
case OpContainedIn:
v = "op-contained-in"
case OpHasKey:
v = "op-has-key"
case OpHasKeyAny:
v = "op-has-key-any"
case OpHasKeyAll:
v = "op-has-key-all"
case OpIsNull:
v = "op-is-null"
}
return fmt.Sprintf("<%s>", v)
}
type ValType int
const (
@ -95,6 +149,7 @@ const (
ValBool
ValList
ValVar
ValNone
)
type AggregrateOp int
@ -320,112 +375,20 @@ func (com *Compiler) compileArgNode(val *Node) (*Exp, error) {
if !ok || eT == nil {
return nil, fmt.Errorf("[Where] unexpected value poped out %v", intf)
}
node := eT.node
if com.bl != nil && com.bl.MatchString(node.Name) {
if len(eT.node.Name) != 0 &&
com.bl != nil && com.bl.MatchString(eT.node.Name) {
continue
}
ex := &Exp{}
ex, err := newExp(st, eT)
name := strings.ToLower(node.Name)
if name[0] == '_' {
name = name[1:]
if err != nil {
return nil, err
}
switch name {
case "and":
ex.Op = OpAnd
pushChildren(st, ex, node)
case "or":
ex.Op = OpOr
pushChildren(st, ex, node)
case "not":
ex.Op = OpNot
pushChildren(st, ex, node)
case "eq", "equals":
ex.Op = OpEquals
ex.Val = node.Val
case "neq", "not_equals":
ex.Op = OpNotEquals
ex.Val = node.Val
case "gt", "greater_than":
ex.Op = OpGreaterThan
ex.Val = node.Val
case "lt", "lesser_than":
ex.Op = OpLesserThan
ex.Val = node.Val
case "gte", "greater_or_equals":
ex.Op = OpGreaterOrEquals
ex.Val = node.Val
case "lte", "lesser_or_equals":
ex.Op = OpLesserOrEquals
ex.Val = node.Val
case "in":
ex.Op = OpIn
setListVal(ex, node)
case "nin", "not_in":
ex.Op = OpNotIn
setListVal(ex, node)
case "like":
ex.Op = OpLike
ex.Val = node.Val
case "nlike", "not_like":
ex.Op = OpNotLike
ex.Val = node.Val
case "ilike":
ex.Op = OpILike
ex.Val = node.Val
case "nilike", "not_ilike":
ex.Op = OpILike
ex.Val = node.Val
case "similar":
ex.Op = OpSimilar
ex.Val = node.Val
case "nsimilar", "not_similar":
ex.Op = OpNotSimilar
ex.Val = node.Val
case "contains":
ex.Op = OpContains
ex.Val = node.Val
case "contained_in":
ex.Op = OpContainedIn
ex.Val = node.Val
case "has_key":
ex.Op = OpHasKey
ex.Val = node.Val
case "has_key_any":
ex.Op = OpHasKeyAny
ex.Val = node.Val
case "has_key_all":
ex.Op = OpHasKeyAll
ex.Val = node.Val
case "is_null":
ex.Op = OpIsNull
ex.Val = node.Val
default:
pushChildren(st, eT.parent, node)
continue // skip node
}
if ex.Op != OpAnd && ex.Op != OpOr && ex.Op != OpNot {
switch node.Type {
case nodeStr:
ex.Type = ValStr
case nodeInt:
ex.Type = ValInt
case nodeBool:
ex.Type = ValBool
case nodeFloat:
ex.Type = ValFloat
case nodeList:
ex.Type = ValList
case nodeVar:
ex.Type = ValVar
default:
return nil, fmt.Errorf("[Where] valid values include string, int, float, boolean and list: %s", node.Type)
}
setWhereColName(ex, node)
if ex == nil {
continue
}
if eT.parent == nil {
@ -563,6 +526,118 @@ func compileSub() (*Query, error) {
return nil, nil
}
func newExp(st *util.Stack, eT *expT) (*Exp, error) {
ex := &Exp{}
node := eT.node
if len(node.Name) == 0 {
pushChildren(st, eT.parent, node)
return nil, nil
}
name := strings.ToLower(node.Name)
if name[0] == '_' {
name = name[1:]
}
switch name {
case "and":
ex.Op = OpAnd
pushChildren(st, ex, node)
case "or":
ex.Op = OpOr
pushChildren(st, ex, node)
case "not":
ex.Op = OpNot
st.Push(&expT{ex, node.Children[0]})
case "eq", "equals":
ex.Op = OpEquals
ex.Val = node.Val
case "neq", "not_equals":
ex.Op = OpNotEquals
ex.Val = node.Val
case "gt", "greater_than":
ex.Op = OpGreaterThan
ex.Val = node.Val
case "lt", "lesser_than":
ex.Op = OpLesserThan
ex.Val = node.Val
case "gte", "greater_or_equals":
ex.Op = OpGreaterOrEquals
ex.Val = node.Val
case "lte", "lesser_or_equals":
ex.Op = OpLesserOrEquals
ex.Val = node.Val
case "in":
ex.Op = OpIn
setListVal(ex, node)
case "nin", "not_in":
ex.Op = OpNotIn
setListVal(ex, node)
case "like":
ex.Op = OpLike
ex.Val = node.Val
case "nlike", "not_like":
ex.Op = OpNotLike
ex.Val = node.Val
case "ilike":
ex.Op = OpILike
ex.Val = node.Val
case "nilike", "not_ilike":
ex.Op = OpILike
ex.Val = node.Val
case "similar":
ex.Op = OpSimilar
ex.Val = node.Val
case "nsimilar", "not_similar":
ex.Op = OpNotSimilar
ex.Val = node.Val
case "contains":
ex.Op = OpContains
ex.Val = node.Val
case "contained_in":
ex.Op = OpContainedIn
ex.Val = node.Val
case "has_key":
ex.Op = OpHasKey
ex.Val = node.Val
case "has_key_any":
ex.Op = OpHasKeyAny
ex.Val = node.Val
case "has_key_all":
ex.Op = OpHasKeyAll
ex.Val = node.Val
case "is_null":
ex.Op = OpIsNull
ex.Val = node.Val
default:
pushChildren(st, eT.parent, node)
return nil, nil // skip node
}
if ex.Op != OpAnd && ex.Op != OpOr && ex.Op != OpNot {
switch node.Type {
case nodeStr:
ex.Type = ValStr
case nodeInt:
ex.Type = ValInt
case nodeBool:
ex.Type = ValBool
case nodeFloat:
ex.Type = ValFloat
case nodeList:
ex.Type = ValList
case nodeVar:
ex.Type = ValVar
default:
return nil, fmt.Errorf("[Where] valid values include string, int, float, boolean and list: %s", node.Type)
}
setWhereColName(ex, node)
}
return ex, nil
}
func setListVal(ex *Exp, node *Node) {
if len(node.Children) != 0 {
switch node.Children[0].Type {