为什么PowerShell工作流比XML文件分析的非工作流脚本慢得多

Question

我正在编写一个PowerShell程序来分析1900+大XML配置文件(50000+行,1.5Mb)的内容.仅用于测试我将36个测试文件移动到我的PC(Win 10; PS 5.1; 32GB RAM)并编写快速脚本来测试执行速度.

$TestDir = "E:\Powershell\Test"
$TestXMLs = Get-ChildItem $TestDir -Recurse -Include *.xml

foreach ($TestXML in $TestXMLs)
{
    [xml]$XML = Get-Content $TestXML
    (($XML.root.servers.server).Where{$_.name -eq "Server1"}).serverid
}

这完成了36到40秒.我用measure-command做了几次测试.

然后我尝试使用foreach -paralell的工作流程,假设并行加载几个文件将给我更快的过程.

Workflow Test-WF
{
    $TestDir = "E:\Powershell\Test"
    $TestXMLs = Get-ChildItem $TestDir -Recurse -Include *.xml

    foreach -parallel -throttle 10 ($TestXML in $TestXMLs)
    {
        [xml]$XML = Get-Content $TestXML
        (($TestXML.root.servers.server).Where{$_.name -eq "Sevrver1"}).serverid
    }
}

Test-WF #execute workflow

具有工作流程的脚本需要118到132秒.

现在我只是想知道工作流程运行得如此之慢的原因是什么？重新编译为XMAL可能或更慢的算法在WWF中加载XML文件？

Answer 1

foreach -parallel 到目前为止,PowerShell是最慢的并行化选项,因为Workflows不是为速度而设计的,而是针对可以安全中断和恢复的长时间运行的操作.

这些安全机制的实现引入了一些开销,这就是为什么在作为工作流运行时脚本速度较慢的原因.

如果要优化执行速度,请改用运行空间:

$TestDir = "E:\Powershell\Test"
$TestXMLs = Get-ChildItem $TestDir -Recurse -Include *.xml

# Set up runspace pool
$RunspacePool = [runspacefactory]::CreateRunspacePool(1,10)
$RunspacePool.Open()

# Assign new jobs/runspaces to a variable
$Runspaces = foreach ($TestXML in $TestXMLs)
{
    # Create new PowerShell instance to hold the code to execute, add arguments
    $PSInstance = [powershell]::Create().AddScript({
        param($XMLPath)

        [xml]$XML = Get-Content $XMLPath
        (($XML.root.servers.server).Where{$_.name -eq "Server1"}).serverid
    }).AddParameter('XMLPath', $TestXML.FullName)

    # Assing PowerShell instance to RunspacePool
    $PSInstance.RunspacePool = $RunspacePool

    # Start executing asynchronously, keep instance + IAsyncResult objects
    New-Object psobject -Property @{
        Instance = $PSInstance
        IAResult = $PSInstance.BeginInvoke()
        Argument = $TestXML
    }
}

# Wait for the the runspace jobs to complete
while($Runspaces |Where-Object{-not $_.IAResult.IsCompleted})
{
    Start-Sleep -Milliseconds 500
}

# Collect the results
$Results = $Runspaces |ForEach-Object {
    $Output = $_.Instance.EndInvoke($_.IAResult)
    New-Object psobject -Property @{
        File = $TestXML
        ServerID = $Output
    }
}

快速XML处理奖金提示:

正如wOxxOm建议的那样,使用Xml.Load()比使用Get-ContentXML文档读取更快.

此外,如果有许多或节点,使用点符号($xml.root.servers.server)和Where({})扩展方法也会非常缓慢.使用带有XPath表达式的方法来搜索"Server1"(请注意XPath是区分大小写的):serversserverSelectNodes()

$PSInstance = [powershell]::Create().AddScript({
    param($XMLPath)

    $XML = New-Object Xml
    $XML.Load($XMLPath)
    $Server1Node = $XML.SelectNodes('/root/servers/server[@name = "Server1"]')
    return $Server1Node.serverid
}).AddParameter('XMLPath', $TestXML.FullName)