Lucene in Action

....add(Field.Keyword("indexDate", new Date())); Internally, Lucene uses the DateField class to convert ... be careful when using this method: Dates converted to indexable Strings by DateField include all ... Email messages include sent and received dates, files have several timestamps associated with them, and HTTP responses have a Last-Modified header that includes the date of the requested page's last modification. Chances are, like many other Lucene users, you'll need to index dates. Lucene comes...

..., or parsing fails. In our example index, the field pubmonth isn't a date field; it's text of the format YYYYMM. Handling date ranges When a range query is encountered, the parser code first attempts to convert the start and end terms to dates. If the terms are valid dates, according to DateFormat. SHORT and lenient parsing within the default or specified locale, then the dates are converted ... Text or date range queries use bracketed syntax, with TO between the beginning term and ending term... [Full sample chapter]

...When returning search hits, Lucene orders them by their score by default. Some- times, however, you need to order results using some other criteria. For instance, if you're searching email messages, you may want to order results by sent or received date, or perhaps by message size. If you want to be able to sort results by a Field value, you must add it as a Field that is indexed but not tokenized (for example, Field.Keyword). Fields used for sorting must be convertible to Integers, Floats...

...To index data with Lucene, you must first convert it to a stream of plain-text tokens, the format that Lucene can digest. In chapter 1, we limited our examples to indexing and searching .txt files, which allowed us to slurp their content and use it to populate Field instances. However, things aren ... at table 1.2, page 21, you'll see that Field methods always take String values and, in some cases, Date ... breaks down into three main operations: converting data to text, analyzing it, and saving it to the index...

..., and so on) of document data is irrelevant to Lucene. The meta-data such as author, title, subject, date ... isn't as straightforward as pro- cessing purely textual content that can easily be converted to a String or Reader... [Full sample chapter]

... the special minimum and maximum dates return strings that must be con- verted to a date representation ... The date field type is covered in section 2.4 along with its caveats. Having a date field, you filter as shown in testDateFilter() in listing 5.6. Our book data indexes the last modified date of each book data file as a modified field, indexed as a Field.Keyword(String, Date). We test the date ... .search(allBooks); numAllBooks = hits.length(); } public void testDateFilter() throws Exception { Date jan...

... original value should be preserved in its entirety, such as URLs, file system paths, dates, personal ... Security numbers, URLs, personal names Field.Keyword(String, Date) Dates Field.UnIndexed(String ... the field's name and its value. In addition, a Keyword field can be passed both a String and a Date ... , the value is converted to a Reader before indexing; these additional methods exist to provide a friendlier... [Full sample chapter]

...QueryParser rears its ugly head again with our changed date format. The built-in date-range handling parses DateFormat.SHORT formats into the DateField text conversions. It would be nice to let users enter a typical date format like 1/1/04 and have it converted to our revised date format of YYYYMMDD. This can be Figure 6.3 Performance test results for 10 concurrent SearchTests, each required ... .getDateInstance(DateFormat.SHORT, getLocale()); df.setLenient(true); Date d1 = df.parse(part1); Date...

.... After a search database is built, it is dynamically kept up to date. There is never a need to spider ... will not find good results. jGuru converts everything to lowercase, strips plurals, strips punctuation...

...Lucene is all about dealing with text. You've seen in several places how dates can be handled, which amounts to their being converted into a text representation that can be ordered alphabetically. Handling numbers is basically the same, except implementing a conversion to a text format is left up to you. In this section, our example scenario indexes an integer id field so that range queries can be performed. If we indexed toString representations of the inte- gers 1 through 10, the order...

...>. * * @param is the InputStream to convert to a Document * @return a ready-to-index instance ... Document instances, not on the actual indexing. After all, once these parsers convert their input ... for all documents if desired (such as last modified date, file sys- tem path, URL, and so on). Now...

... instance long start = new Date().getTime(); indexer.index(writer, new File(args[1])); j First call to index method writer.optimize(); writer.close(); 1) Optimize long end = new Date().getTime(); index ... into an instance of Properties. h The index to which all Files converted to Lucene Documents...

... the index to be updated more frequently, which makes it more up to date but also slows down the indexing ... consider converting your index to the compound structure. As described in appendix B, doing...

... 27 Fragmenter 300 converting native files to DNA 354 FrenchAnalyzer 282 ASCII 142 Docco 265 fuzzy string ... Berkeley UTF-8 140 DB 307 Etymon PJ 264 H date, indexing 216 Explanation 80 DateField 39 Harvest ... 291 converting 400-401 RTF documents 252-253 query validation 291 performance comparison ... 11 IndexReader 199 Konrad, Karsten 344 dates 39-40, 216 deleting documents 375 Korean analysis ... date parsing 218 analyzers 284 with SpanNearQuery 166 date parsing locale 97 building components...