For as long as I’ve been using neo4j‘s cypher query language Michael has been telling me to use parameters in my queries but the performance of the queries was always acceptable so I didn’t feel the need. However, recently I was playing around with a data set and I created ~500 nodes using code similar to this:
require 'open-uri'
open("data/people.cyp", 'w') { |f|
(1..500).each do |value|
f.puts("CREATE (p:Person{name: \"#{value}\"})")
end
}That creates a file of cypher statements that look like this:
CREATE (:Person{name: "person1"})
CREATE (:Person{name: "person2"})
CREATE (:Person{name: "person3"})
CREATE (:Person{name: "person4"})
CREATE (:Person{name: "person5"})
...If we execute those statements in the neo4j-shell or web admin we get the following output:
==> +-------------------+ ==> | No data returned. | ==> +-------------------+ ==> Nodes created: 500 ==> Properties set: 500 ==> Labels added: 500 ==> 27706 ms
As far as I understand the reason it takes that long is that cypher creates and then caches a query plan for each create statement because it has no way of knowing that they are effectively the same. Since I was deleting/reloading the data quite frequently I wanted to make my feedback loop a bit quicker so it was finally time to apply Michael’s advice. The way that parameters work is that you add placeholders into your cypher queries and then provide values that should reify those placeholders when you execute your query. In this case it seemed easiest to make use of neography to execute my query against a running neo4j server. If we want to create a single person with a parameterised name then we’d write the following code:
require 'neography'
params = { :name => "Mark" }
Neography::Rest.new.execute_query("CREATE (:Person {name: {name} })", params)If we run a query to get back all people we can see they’ve been successfully created:
neo4j-sh (0)$ match p:Person return p;
==> +----------------------+
==> | p |
==> +----------------------+
==> | Node[1]{name:"Mark"} |
==> +----------------------+
==> 1 row
==> 175 msI wanted to pass in an array of names which I initially thought I could do but changing the value in the params hash to an array:
require 'neography'
params = { :name => [] }
(1..500).each do |value|
params[:name] << "person#{value}"
end
Neography::Rest.new.execute_query("CREATE (:Person {name: {name} })", params)I ran a query to get back my 500 people but only got one back:
neo4j-sh (0)$ MATCH p:Person RETURN p
> ;
==> +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | p |
==> +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> | Node[1]{name:["person1","person2","person3","person4","person5","person6","person7","person8","person9","person10","person11","person12","person13","person14","person15","person16","person17","person18","person19","person20","person21","person22","person23","person24","person25","person26","person27","person28","person29","person30","person31","person32","person33","person34","person35","person36","person37","person38","person39","person40","person41","person42","person43","person44","person45","person46","person47","person48","person49","person50","person51","person52","person53","person54","person55","person56","person57","person58","person59","person60","person61","person62","person63","person64","person65","person66","person67","person68","person69","person70","person71","person72","person73","person74","person75","person76","person77","person78","person79","person80","person81","person82","person83","person84","person85","person86","person87","person88","person89","person90","person91","person92","person93","person94","person95","person96","person97","person98","person99","person100","person101","person102","person103","person104","person105","person106","person107","person108","person109","person110","person111","person112","person113","person114","person115","person116","person117","person118","person119","person120","person121","person122","person123","person124","person125","person126","person127","person128","person129","person130","person131","person132","person133","person134","person135","person136","person137","person138","person139","person140","person141","person142","person143","person144","person145","person146","person147","person148","person149","person150","person151","person152","person153","person154","person155","person156","person157","person158","person159","person160","person161","person162","person163","person164","person165","person166","person167","person168","person169","person170","person171","person172","person173","person174","person175","person176","person177","person178","person179","person180","person181","person182","person183","person184","person185","person186","person187","person188","person189","person190","person191","person192","person193","person194","person195","person196","person197","person198","person199","person200","person201","person202","person203","person204","person205","person206","person207","person208","person209","person210","person211","person212","person213","person214","person215","person216","person217","person218","person219","person220","person221","person222","person223","person224","person225","person226","person227","person228","person229","person230","person231","person232","person233","person234","person235","person236","person237","person238","person239","person240","person241","person242","person243","person244","person245","person246","person247","person248","person249","person250","person251","person252","person253","person254","person255","person256","person257","person258","person259","person260","person261","person262","person263","person264","person265","person266","person267","person268","person269","person270","person271","person272","person273","person274","person275","person276","person277","person278","person279","person280","person281","person282","person283","person284","person285","person286","person287","person288","person289","person290","person291","person292","person293","person294","person295","person296","person297","person298","person299","person300","person301","person302","person303","person304","person305","person306","person307","person308","person309","person310","person311","person312","person313","person314","person315","person316","person317","person318","person319","person320","person321","person322","person323","person324","person325","person326","person327","person328","person329","person330","person331","person332","person333","person334","person335","person336","person337","person338","person339","person340","person341","person342","person343","person344","person345","person346","person347","person348","person349","person350","person351","person352","person353","person354","person355","person356","person357","person358","person359","person360","person361","person362","person363","person364","person365","person366","person367","person368","person369","person370","person371","person372","person373","person374","person375","person376","person377","person378","person379","person380","person381","person382","person383","person384","person385","person386","person387","person388","person389","person390","person391","person392","person393","person394","person395","person396","person397","person398","person399","person400","person401","person402","person403","person404","person405","person406","person407","person408","person409","person410","person411","person412","person413","person414","person415","person416","person417","person418","person419","person420","person421","person422","person423","person424","person425","person426","person427","person428","person429","person430","person431","person432","person433","person434","person435","person436","person437","person438","person439","person440","person441","person442","person443","person444","person445","person446","person447","person448","person449","person450","person451","person452","person453","person454","person455","person456","person457","person458","person459","person460","person461","person462","person463","person464","person465","person466","person467","person468","person469","person470","person471","person472","person473","person474","person475","person476","person477","person478","person479","person480","person481","person482","person483","person484","person485","person486","person487","person488","person489","person490","person491","person492","person493","person494","person495","person496","person497","person498","person499","person500"]} |
==> +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
==> 1 rowI’d actually made use of parameters in such a way that only one person got created and had their ‘name’ property set to an array of 500 names! I tweaked the code to pass in an array of maps instead of strings like so:
require 'neography'
params = { :people => [] }
(1..500).each do |value|
params[:people] << { :name => "person#{value}"}
end
Neography::Rest.new.execute_query("CREATE (:Person {people})", params)I ran my script to populate the database before querying it again:
neo4j-sh (0)$ MATCH p:Person RETURN p;
==> +-----------------------------+
==> | p |
==> +-----------------------------+
==> | Node[1]{name:"person1"} |
==> | Node[2]{name:"person2"} |
==> | Node[3]{name:"person3"} |
==> | Node[4]{name:"person4"} |
==> | Node[5]{name:"person5"} |
==> | Node[6]{name:"person6"} |
==> | Node[7]{name:"person7"} |
==> | Node[8]{name:"person8"} |
==> | Node[9]{name:"person9"} |
==> | Node[10]{name:"person10"} |
==> | Node[11]{name:"person11"} |
...
==> +-----------------------------+
==> 500 rows
==> 1081 msThis time I got the expected result so the only thing to do was to quickly check how long it had taken to get the parameterised data into the database:
$ time bundle exec ruby people.rb real 0m0.993s user 0m0.733s sys 0m0.097s
And the winner is…Michael, by about 26 seconds per run.
In this version cypher creates a query plan the first time it encounters the CREATE statement but after that it can use the query plan for the subsequent calls which leads to a massive performance improvement.
Alistair pointed out that the query parameter approach is quite nice when used in applications because you don’t need to worry about string concatenation, you can just change values in maps instead.
Based on this experience I can now confirm Max De Marzi‘s tweet from last week:
Public Service Announcement for folks new to @neo4j and Cypher: Please use parameters in your queries http://docs.neo4j.org/chunked/milestone/cypher-parameters.html #MuchFaster
Reference: neo4j/cypher: Getting the hang of query parameters from our JCG partner Mark Needham at the Mark Needham Blog blog.
