逐行读取大文件

Question

我正在尝试根据Clojure中的迭代为大文件编写阅读器.但是我怎么能在Clojure中逐行返回？我想做那样的事情:

(println(do_something(readFile(:file opts)));处理并打印第一行
(println(do_something(readFile(:file opts)));处理并打印第二行

码:

(ns testapp.core
  (:gen-class)
  (:require [clojure.tools.cli :refer [cli]])
  (:require [clojure.java.io]))


(defn readFile [file, cnt]
  ; Iterate over opened file (read line by line)
  (with-open [rdr (clojure.java.io/reader file)]
    (let [seq (line-seq rdr)]
      ; how return only one line there? and after, when needed, take next line?
    )))

(defn -main [& args]
  ; Main function for project 
  (let [[opts args banner] 
        (cli args
          ["-h" "--help" "Print this help" :default false :flag true]
          ["-f" "--file" "REQUIRED: File with data"]
          ["-c" "--clusters" "Count of clusters" :default 3]
          ["-g" "--hamming" "Use Hamming algorithm"]
          ["-e" "--evklid" "Use Evklid algorithm"]
          )]
    ; Print help, when no typed args
    (when (:help opts)
      (println banner)
      (System/exit 0))
    ; Or process args and start work
    (if (and (:file opts) (or (:hamming opts) (:evklid opts)))
      (do
        ; Use Hamming algorithm
        (if (:hamming opts)
          (do
            (println (readFile (:file opts))
            (println (readFile (:file opts))
          )
          ;(count (readFile (:file opts)))
        ; Use Evklid algorithm
        (println "Evklid")))
      (println "Please, type path for file and algorithm!")))) 

Answer 1

可能是我不明白你的意思是"逐行退货",但我建议你写函数,它接受文件和处理功能,然后为你的大文件的每一行打印处理功能的结果.或者,更普遍的方式,让我们接受处理功能和输出功能(默认为println),所以如果我们不仅要打印,而是通过网络发送,保存到某个地方,发送到另一个线程等:

(defn process-file-by-lines
  "Process file reading it line-by-line"
  ([file]
   (process-file-by-lines file identity))
  ([file process-fn]
   (process-file-by-lines file process-fn println))
  ([file process-fn output-fn]
   (with-open [rdr (clojure.java.io/reader file)]
     (doseq [line (line-seq rdr)]
       (output-fn
         (process-fn line))))))

所以

(process-file-by-lines "/tmp/tmp.txt") ;; Will just print file line by ine
(process-file-by-lines "/tmp/tmp.txt"
                       reverse) ;; Will print each line reversed

Answer 2

尝试剂量：

(defn readFile [file]
  (with-open [rdr (clojure.java.io/reader file)]
    (doseq [line (line-seq rdr)]
      (println line))))

Answer 3

您还可以尝试从阅读器中延迟读取，这与 . 返回的延迟字符串列表不同line-seq。在这个非常相似的问题的答案中讨论了详细信息，但要点如下：

 (defn lazy-file-lines [file]
      (letfn [(helper [rdr]
                (lazy-seq
                  (if-let [line (.readLine rdr)]
                    (cons line (helper rdr))
                    (do (.close rdr) nil))))]
        (helper (clojure.java.io/reader file))))

然后，您可以map跳过这些行，这些行只会在必要时阅读。正如链接答案中详细讨论的那样，缺点是，如果您不阅读直到文件末尾，则(.close rdr)永远不会运行，可能会导致资源问题。