有关使用fs.readdir进行异步目录搜索的任何想法吗？我意识到我们可以引入递归并调用读取目录函数与下一个目录来读取,但有点担心它不是异步...
有任何想法吗？我看过node-walk这很棒,但是不给我一个数组中的文件,比如readdir.虽然
寻找像...的输出
['file1.txt', 'file2.txt', 'dir/file3.txt']
chj*_*hjj 361
基本上有两种方法可以实现这一点.在异步环境中,您会注意到有两种循环:串行和并行.串行循环在进入下一次迭代之前等待一次迭代完成 - 这保证循环的每次迭代按顺序完成.在并行循环中,所有迭代都是同时启动的,并且可以在另一个迭代之前完成,但是,它比串行循环快得多.因此,在这种情况下,使用并行循环可能更好,因为无论步行完成的顺序无关紧要,只要它完成并返回结果(除非您按顺序要求它们).
并行循环看起来像这样:
var fs = require('fs');
var path = require('path');
var walk = function(dir, done) {
var results = [];
fs.readdir(dir, function(err, list) {
if (err) return done(err);
var pending = list.length;
if (!pending) return done(null, results);
list.forEach(function(file) {
file = path.resolve(dir, file);
fs.stat(file, function(err, stat) {
if (stat && stat.isDirectory()) {
walk(file, function(err, res) {
results = results.concat(res);
if (!--pending) done(null, results);
});
} else {
results.push(file);
if (!--pending) done(null, results);
}
});
});
});
};
串行循环看起来像这样:
var fs = require('fs');
var path = require('path');
var walk = function(dir, done) {
var results = [];
fs.readdir(dir, function(err, list) {
if (err) return done(err);
var i = 0;
(function next() {
var file = list[i++];
if (!file) return done(null, results);
file = path.resolve(dir, file);
fs.stat(file, function(err, stat) {
if (stat && stat.isDirectory()) {
walk(file, function(err, res) {
results = results.concat(res);
next();
});
} else {
results.push(file);
next();
}
});
})();
});
};
并在您的主目录中测试它(警告:如果您的主目录中有很多东西,结果列表将是巨大的):
walk(process.env.HOME, function(err, results) {
if (err) throw err;
console.log(results);
});
编辑:改进的例子.
小智 99
为了防止任何人发现它有用,我还整理了一个同步版本.
var walk = function(dir) {
var results = [];
var list = fs.readdirSync(dir);
list.forEach(function(file) {
file = dir + '/' + file;
var stat = fs.statSync(file);
if (stat && stat.isDirectory()) {
/* Recurse into a subdirectory */
results = results.concat(walk(file));
} else {
/* Is a file */
results.push(file);
}
});
return results;
}
提示:过滤时使用较少的资源.在此功能本身内过滤.例如,results.push(file);用以下代码替换.根据需要调整:
file_type = file.split(".").pop();
file_name = file.split(/(\\|\/)/g).pop();
if (file_type == "json") results.push(file);
qwt*_*tel 88
这个使用了节点8中可用的最新的buzzwordy功能,包括Promises,util/promisify,destructuring,async-await,map + reduce等等,让你的同事在试图弄清楚正在进行.
没有外部依赖.
const { promisify } = require('util');
const { resolve } = require('path');
const fs = require('fs');
const readdir = promisify(fs.readdir);
const stat = promisify(fs.stat);
async function getFiles(dir) {
const subdirs = await readdir(dir);
const files = await Promise.all(subdirs.map(async (subdir) => {
const res = resolve(dir, subdir);
return (await stat(res)).isDirectory() ? getFiles(res) : res;
}));
return files.reduce((a, f) => a.concat(f), []);
}
用法:
getFiles(__dirname)
.then(files => console.log(files))
.catch(e => console.error(e));
更新了节点10+以及更多的whizbang:
const { resolve } = require('path');
const { readdir } = require('fs').promises;
async function getFiles(dir) {
const dirents = await readdir(dir, { withFileTypes: true });
const files = await Promise.all(dirents.map((dirent) => {
const res = resolve(dir, dirent.name);
return dirent.isDirectory() ? getFiles(res) : res;
}));
return Array.prototype.concat(...files);
}
如果你想让每个人都完全放心,你可以使用异步迭代器来使用以下版本.除了非常酷,它还允许消费者一次一个地提取结果,使其更适合真正大的目录.
const { resolve } = require('path');
const { readdir } = require('fs').promises;
async function* getFiles(dir) {
const dirents = await readdir(dir, { withFileTypes: true });
for (const dirent of dirents) {
const res = resolve(dir, dirent.name);
if (dirent.isDirectory()) {
yield* getFiles(res);
} else {
yield res;
}
}
}
用法已更改,因为返回类型现在是异步迭代器而不是promise
(async () => {
for await (const f of getFiles('.')) {
console.log(f);
}
})()
Joh*_*sen 86
A.看一下文件模块.它有一个叫做walk的函数:
file.walk(开始,回调)
导航文件树,为每个目录调用回调,传入(null,dirPath,dirs,files).
这可能适合你!是的,它是异步的.但是,如果您需要,我认为您必须自己聚合完整路径.
B.另一种选择,甚至是我最喜欢的一种:使用unix find.为什么要再做一次,已经编程了？也许不完全是你需要的,但仍然值得一试:
var execFile = require('child_process').execFile;
execFile('find', [ 'somepath/' ], function(err, stdout, stderr) {
var file_list = stdout.split('\n');
/* now you've got a list with full path file names */
});
Find有一个很好的内置缓存机制,只要只有少数文件夹发生变化,后续搜索速度非常快.
Mir*_*ili 36
import {readdir} from 'node:fs/promises'
import {join} from 'node:path'
const walk = async (dirPath) => Promise.all(
await readdir(dirPath, { withFileTypes: true }).then((entries) => entries.map((entry) => {
const childPath = join(dirPath, entry.name)
return entry.isDirectory() ? walk(childPath) : childPath
})),
)
特别感谢Function的提示：{withFileTypes: true}.
这会自动保留源目录的树结构（您可能需要）。例如如果：
const allFiles = await walk('src')
那么allFiles就会是一棵像这样的树：
[
[
'src/client/api.js',
'src/client/http-constants.js',
'src/client/index.html',
'src/client/index.js',
[ 'src/client/res/favicon.ico' ],
'src/client/storage.js'
],
[ 'src/crypto/keygen.js' ],
'src/discover.js',
[
'src/mutations/createNewMutation.js',
'src/mutations/newAccount.js',
'src/mutations/transferCredit.js',
'src/mutations/updateApp.js'
],
[
'src/server/authentication.js',
'src/server/handlers.js',
'src/server/quick-response.js',
'src/server/server.js',
'src/server/static-resources.js'
],
[ 'src/util/prompt.js', 'src/util/safeWriteFile.js' ],
'src/util.js'
]
allFiles.flat(Number.POSITIVE_INFINITY)
[
'src/client/api.js',
'src/client/http-constants.js',
'src/client/index.html',
'src/client/index.js',
'src/client/res/favicon.ico',
'src/client/storage.js',
'src/crypto/keygen.js',
'src/discover.js',
'src/mutations/createNewMutation.js',
'src/mutations/newAccount.js',
'src/mutations/transferCredit.js',
'src/mutations/updateApp.js',
'src/server/authentication.js',
'src/server/handlers.js',
'src/server/quick-response.js',
'src/server/server.js',
'src/server/static-resources.js',
'src/util/prompt.js',
'src/util/safeWriteFile.js',
'src/util.js'
]
Dio*_*oso 27
我建议使用node-glob来完成该任务.
var glob = require( 'glob' );
glob( 'dirname/**/*.js', function( err, files ) {
console.log( files );
});
Dom*_*nic 14
如果你想使用npm包,扳手是非常好的.
var wrench = require("wrench");
var files = wrench.readdirSyncRecursive("directory");
wrench.readdirRecursive("directory", function (error, files) {
// live your dreams
});
编辑(2018年):
最近阅读的人:作者在2015年弃用了这个包:
不推荐使用wrench.js,并且在相当长的一段时间内没有更新.我强烈建议使用fs-extra来执行任何额外的文件系统操作.
我喜欢的答案,从chjj以上,并且不会已经能够没有这种开始创建我的版本并行循环的.
var fs = require("fs");
var tree = function(dir, done) {
var results = {
"path": dir
,"children": []
};
fs.readdir(dir, function(err, list) {
if (err) { return done(err); }
var pending = list.length;
if (!pending) { return done(null, results); }
list.forEach(function(file) {
fs.stat(dir + '/' + file, function(err, stat) {
if (stat && stat.isDirectory()) {
tree(dir + '/' + file, function(err, res) {
results.children.push(res);
if (!--pending){ done(null, results); }
});
} else {
results.children.push({"path": dir + "/" + file});
if (!--pending) { done(null, results); }
}
});
});
});
};
module.exports = tree;
我也创造了一个要点.欢迎评论.我仍然在NodeJS领域开始,所以这是我希望了解更多的一种方式.
我在这个帖子中没有找到我想要的答案；不同的答案中有一些相似的元素，但我只是想要一些简单易读的东西。
以防万一它对将来的任何人（即几个月后的我自己）有帮助，这就是我最终使用的：
const { readdir } = require('fs/promises');
const { join } = require('path');
const readdirRecursive = async dir => {
const files = await readdir( dir, { withFileTypes: true } );
const paths = files.map( async file => {
const path = join( dir, file.name );
if ( file.isDirectory() ) return await readdirRecursive( path );
return path;
} );
return ( await Promise.all( paths ) ).flat( Infinity );
}
module.exports = {
readdirRecursive,
}
使用node-dir准确生成您喜欢的输出
var dir = require('node-dir');
dir.files(__dirname, function(err, files) {
if (err) throw err;
console.log(files);
//we have an array of files now, so now we can iterate that array
files.forEach(function(path) {
action(null, path);
})
});
这是一个简单的同步递归解决方案
const fs = require('fs')
const getFiles = path => {
const files = []
for (const file of fs.readdirSync(path)) {
const fullPath = path + '/' + file
if(fs.lstatSync(fullPath).isDirectory())
getFiles(fullPath).forEach(x => files.push(file + '/' + x))
else files.push(file)
}
return files
}
用法：
const files = getFiles(process.cwd())
console.log(files)
您可以异步编写它，但没有必要。只需确保输入目录存在并且可访问。
现代基于承诺的读取目录递归版本：
const fs = require('fs');
const path = require('path');
const readDirRecursive = async (filePath) => {
const dir = await fs.promises.readdir(filePath);
const files = await Promise.all(dir.map(async relativePath => {
const absolutePath = path.join(filePath, relativePath);
const stat = await fs.promises.lstat(absolutePath);
return stat.isDirectory() ? readDirRecursive(absolutePath) : absolutePath;
}));
return files.flat();
}
随着递归
var fs = require('fs')
var path = process.cwd()
var files = []
var getFiles = function(path, files){
fs.readdirSync(path).forEach(function(file){
var subpath = path + '/' + file;
if(fs.lstatSync(subpath).isDirectory()){
getFiles(subpath, files);
} else {
files.push(path + '/' + file);
}
});
}
调用
getFiles(path, files)
console.log(files) // will log all files in directory
异步
const fs = require('fs')
const path = require('path')
const readdir = (p, done, a = [], i = 0) => fs.readdir(p, (e, d = []) =>
d.map(f => readdir(a[a.push(path.join(p, f)) - 1], () =>
++i == d.length && done(a), a)).length || done(a))
readdir(__dirname, console.log)
同步
const fs = require('fs')
const path = require('path')
const readdirSync = (p, a = []) => {
if (fs.statSync(p).isDirectory())
fs.readdirSync(p).map(f => readdirSync(a[a.push(path.join(p, f)) - 1], a))
return a
}
console.log(readdirSync(__dirname))
异步可读
function readdir (currentPath, done, allFiles = [], i = 0) {
fs.readdir(currentPath, function (e, directoryFiles = []) {
if (!directoryFiles.length)
return done(allFiles)
directoryFiles.map(function (file) {
var joinedPath = path.join(currentPath, file)
allFiles.push(joinedPath)
readdir(joinedPath, function () {
i = i + 1
if (i == directoryFiles.length)
done(allFiles)}
, allFiles)
})
})
}
readdir(__dirname, console.log)
注意：两个版本都将遵循符号链接（与原始版本相同fs.readdir）
import { resolve } from 'path';
import { readdir } from 'fs/promises';
async function* getFiles(dir: string): AsyncGenerator<string> {
const entries = await readdir(dir, { withFileTypes: true });
for (const entry of entries) {
const res = resolve(dir, entry.name);
if (entry.isDirectory()) {
yield* getFiles(res);
} else {
yield res;
}
}
}
小智 6
const fs = require('fs/promises');
const getDirRecursive = async (dir) => {
try {
const items = await fs.readdir(dir);
let files = [];
for (const item of items) {
if ((await fs.lstat(`${dir}/${item}`)).isDirectory()) files = [...files, ...(await getDirRecursive(`${dir}/${item}`))];
else files.push({file: item, path: `${dir}/${item}`, parents: dir.split("/")});
}
return files;
} catch (e) {
return e
}
};
用法：await getDirRecursive("./public");
v20.1版本中提供的最短的本机解决方案：
import fs from 'node:fs'
const results = fs.promises.readdir('/tmp', { recursive: true })
recursive选项也受fs.readdir和fs.readdirSync函数支持。
使用 async/await，这应该可以工作：
const FS = require('fs');
const readDir = promisify(FS.readdir);
const fileStat = promisify(FS.stat);
async function getFiles(dir) {
let files = await readDir(dir);
let result = files.map(file => {
let path = Path.join(dir,file);
return fileStat(path).then(stat => stat.isDirectory() ? getFiles(path) : path);
});
return flatten(await Promise.all(result));
}
function flatten(arr) {
return Array.prototype.concat(...arr);
}
您可以使用bluebird.Promisify或这个：
/**
* Returns a function that will wrap the given `nodeFunction`. Instead of taking a callback, the returned function will return a promise whose fate is decided by the callback behavior of the given node function. The node function should conform to node.js convention of accepting a callback as last argument and calling that callback with error as the first argument and success value on the second argument.
*
* @param {Function} nodeFunction
* @returns {Function}
*/
module.exports = function promisify(nodeFunction) {
return function(...args) {
return new Promise((resolve, reject) => {
nodeFunction.call(this, ...args, (err, data) => {
if(err) {
reject(err);
} else {
resolve(data);
}
})
});
};
};
Node 8+内置了 Promisify
请参阅我的其他答案，了解可以更快给出结果的生成器方法。
| 归档时间： |
|
| 查看次数： |
239861 次 |
| 最近记录： |