{"id":364,"date":"2014-07-30T09:04:32","date_gmt":"2014-07-30T07:04:32","guid":{"rendered":"http:\/\/dev.flauschig.ch\/wordpress\/?p=364"},"modified":"2014-07-31T09:36:49","modified_gmt":"2014-07-31T07:36:49","slug":"luke-with-elasticsearch-index","status":"publish","type":"post","link":"http:\/\/dev.flauschig.ch\/wordpress\/?p=364","title":{"rendered":"How to open an Elasticsearch Index with Luke"},"content":{"rendered":"

I’ve started working with Elasticsearch<\/a> and since I know Lucene internals quite well, I want to have a look at the files that Elasticsearch generates.<\/p>\n

There’s a tool called Luke (Lucene Index Toolbox) to do that. There are multiple repositories with Luke:<\/p>\n

    \n
  1. Original<\/a> (abandoned?)<\/li>\n
  2. Dmitry Kan Fork<\/a><\/li>\n
  3. \u013dubo\u0161 Ko\u0161\u010do Fork<\/a><\/li>\n<\/ol>\n

    But since Elasticsearch uses it’s own codecs, it’s not possible to directly open the index with one of these Lukes.<\/p>\n

    To overcome this, the codecs need to be added to Luke.<\/p>\n

    I found an easy tutorial<\/a> for this on Ross Simpsons blog<\/a>.<\/p>\n

    Here’s a copy of the how-to:
    \n1. Clone Dmitry\u2019s Mavenized fork:<\/p>\n

    $ git clone https:\/\/github.com\/DmitryKey\/luke\/<\/pre>\n
    2. Add a dependency on your required version of Elasticsearch to the Luke project\u2019s pom file:<\/p>\n
    \r\n\r\n\r\n    org.elasticsearch<\/groupId>\r\n    elasticsearch<\/artifactId>\r\n    1.3.0<\/version>\r\n<\/dependency>\r\n<\/pre>\n
    3. Compile the Luke jar file (creates target\/luke-with-deps.jar):<\/p>\n
    \r\n$ mvn package\r\n<\/pre>\n
    4. Unpack Luke\u2019s list of known postings formats to a temporary file:<\/p>\n
    \r\n$ unzip target\/luke-with-deps.jar META-INF\/services\/org.apache.lucene.codecs.PostingsFormat -d .\/tmp\/\r\nArchive:  target\/luke-with-deps.jar\r\n  inflating: .\/tmp\/META-INF\/services\/org.apache.lucene.codecs.PostingsFormat\r\n<\/pre>\n
    5. Add the ElasticSearch postings formats to the temp file:<\/p>\n
    \r\n$ echo \"org.elasticsearch.index.codec.postingsformat.BloomFilterPostingsFormat\" \\\r\n    >> .\/tmp\/META-INF\/services\/org.apache.lucene.codecs.PostingsFormat\r\n$ echo \"org.elasticsearch.index.codec.postingsformat.Elasticsearch090PostingsFormat\" \\\r\n    >> .\/tmp\/META-INF\/services\/org.apache.lucene.codecs.PostingsFormat\r\n$ echo \"org.elasticsearch.search.suggest.completion.Completion090PostingsFormat\" \\\r\n    >> .\/tmp\/META-INF\/services\/org.apache.lucene.codecs.PostingsFormat\\\r\n<\/pre>\n
    6. Repack the modified file back into the jar:<\/p>\n
    \r\n$ jar -uf target\/luke-with-deps.jar -C tmp\/ META-INF\/services\/org.apache.lucene.codecs.PostingsFormat\r\n<\/pre>\n
    7. Run Luke<\/p>\n
    \r\n$ .\/luke.sh\r\n<\/pre>\n
    That’s it.
    \nAnd for all the lazy people out there, here’s the prepared     Luke 4.9.0 with dependencies for Elasticsearch 1.3.0 (luke-with-deps-es-4.9.0).<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
    I’ve started working with Elasticsearch and since I know Lucene internals quite well, I want to have a look at the files that Elasticsearch generates. There’s a tool called Luke (Lucene Index Toolbox) to do that. There are multiple repositories with Luke: Original (abandoned?) Dmitry Kan Fork \u013dubo\u0161 Ko\u0161\u010do Fork But since Elasticsearch uses it’s … <\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[3],"tags":[],"class_list":{"0":"entry","1":"post","2":"publish","3":"author-roemer","4":"post-364","6":"format-standard","7":"category-programming"},"acf":[],"views":5796,"_links":{"self":[{"href":"http:\/\/dev.flauschig.ch\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/364"}],"collection":[{"href":"http:\/\/dev.flauschig.ch\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/dev.flauschig.ch\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/dev.flauschig.ch\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/dev.flauschig.ch\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=364"}],"version-history":[{"count":0,"href":"http:\/\/dev.flauschig.ch\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/364\/revisions"}],"wp:attachment":[{"href":"http:\/\/dev.flauschig.ch\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=364"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/dev.flauschig.ch\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=364"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/dev.flauschig.ch\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=364"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}