Dan*_*iel 5 parallel-processing powershell foreach-object
第一次在这里提问。请善待:)
我试图以并行方式递归地获取所有目录,希望减少遍历驱动器所需的时间。下面是我尝试过的代码。本质上我想要做的是输入一个文件夹并对其子文件夹及其子文件夹等并行执行相同的操作,但该函数在并行块内无法识别
function New-RecursiveDirectoryList {
[CmdletBinding()]
param (
# Specifies a path to one or more locations.
[Parameter(Mandatory = $true,
Position = 0,
ValueFromPipeline = $true,
ValueFromPipelineByPropertyName = $true,
HelpMessage = 'Path to one or more locations.')]
[Alias('PSPath')]
[ValidateNotNullOrEmpty()]
[string[]]
$Path
)
process {
foreach ($aPath in $Path) {
Get-Item $aPath
Get-ChildItem -Path $aPath -Directory |
# Recursively call itself in Parallel block not working
# Getting error "The term 'New-RecursiveDirectoryList' is not recognized as a name of a cmdlet"
# Without -Parallel switch this works as expected
ForEach-Object -Parallel {
$_ | New-RecursiveDirectoryList
}
}
}
}
Run Code Online (Sandbox Code Playgroud)
错误:
New-RecursiveDirectoryList:
Line |
2 | $_ | New-RecursiveDirectoryList
| ~~~~~~~~~~~~~~~~~~~~~~~~~~
| The term 'New-RecursiveDirectoryList' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
Run Code Online (Sandbox Code Playgroud)
我也尝试使用 mklement0 提供的解决方案,但没有成功。以下是我的尝试:
function CustomFunction {
[CmdletBinding()]
param (
# Specifies a path to one or more locations.
[Parameter(Mandatory = $true,
Position = 0,
ValueFromPipeline = $true,
ValueFromPipelineByPropertyName = $true,
HelpMessage = 'Path to one or more locations.')]
[Alias('PSPath')]
[ValidateNotNullOrEmpty()]
[string[]]
$Path
)
begin {
# Get the function's definition *as a string*
$funcDef = $function:CustomFunction.ToString()
}
process {
foreach ($aPath in $Path) {
Get-Item $aPath
Get-ChildItem -Path $aPath -Directory |
# Recursively call itself in Parallel block not working
# Getting error "The term 'New-RecursiveDirectoryList' is not recognized as a name of a cmdlet"
# Without -Parallel switch this works as expected
ForEach-Object -Parallel {
$function:CustomFunction = $using:funcDef
$_ | CustomFuction
}
}
}
}
Run Code Online (Sandbox Code Playgroud)
错误
CustomFuction:
Line |
3 | $_ | CustomFuction
| ~~~~~~~~~~~~~
| The term 'CustomFuction' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
Run Code Online (Sandbox Code Playgroud)
有谁知道如何实现这一点或以不同的方式实现这一点?
所以,这对我有用,但显然看起来不太漂亮。需要注意的一件事是,foreach ($aPath in $Path) {...}脚本中的 是不必要的,process {...}当您传递多个路径时,该块将为您处理它。
function Test {
[CmdletBinding()]
param (
# Specifies a path to one or more locations.
[Parameter(
Mandatory,
ParameterSetName = 'LiteralPath',
ValueFromPipelineByPropertyName,
Position = 0)]
[Alias('PSPath')]
[string[]] $LiteralPath
)
begin {
$scriptblock = $MyInvocation.MyCommand.ScriptBlock.ToString()
}
process {
# Get-Item $Path <= This will slow down the script
$LiteralPath | Get-ChildItem -Directory | ForEach-Object -Parallel {
$sb = $using:scriptblock
$def = [scriptblock]::Create($sb)
$_ # You can do this instead
$_ | & $def
}
}
}
Run Code Online (Sandbox Code Playgroud)
回顾这个答案,我今天建议的是不要使用递归,而使用 aConcurrentStack<T>代替,这会提高效率并消耗更少的内存。另外值得注意的是,正如mklement0在他的评论中指出的那样,您的代码一开始就是正确的,问题是由于拼写错误造成的:$_ | CustomFuction->$_ | CustomFunction。
function Test {
[CmdletBinding()]
param (
[Parameter(
Mandatory,
ParameterSetName = 'LiteralPath',
ValueFromPipelineByPropertyName,
Position = 0)]
[Alias('PSPath')]
[string[]] $LiteralPath,
[Parameter()]
[ValidateRange(1, 64)]
[int] $ThrottleLimit = 5
)
begin {
$stack = [System.Collections.Concurrent.ConcurrentStack[System.IO.DirectoryInfo]]::new()
$dir = $null
}
process {
$stack.PushRange($LiteralPath)
while ($stack.TryPop([ref] $dir)) {
$dir | Get-ChildItem -Directory | ForEach-Object -Parallel {
$stack = $using:stack
$stack.Push($_)
$_
} -ThrottleLimit $ThrottleLimit
}
}
}
Run Code Online (Sandbox Code Playgroud)
| 归档时间: |
|
| 查看次数: |
1258 次 |
| 最近记录: |