The context
Sometimes you need to process data coming from various parts of a tree.
For example:
- Different folders of one or several volumes
C:\Program Files\App1\Logs D:\App2\Logs E:\Logs
- Different OUs of one or several Active Directories
OU=Management,DC=MyDomain,DC=Com OU=InternalUsers,DC=MyDomain,DC=Com OU=ExternalUsers,DC=MyDomain,DC=Com
Once the data is collected, you have two options for the processing part, each one having its pros and cons.
- Repeat the processing for each source of collected data
- The code executes faster because the data collection is only processed once
- You have a redundancy of code because you repeat the code for each collection of data
- Concatenate all data into one array and process the whole in one bunch
- The code execution is slower because you preprocess the data to create a single collection
- The code is optimized, cleaner and easier to maintain
Concatenation methods
There are many ways to concatenate arrays.
Here are some of the most popular and well-known methods.
Note: I will not comment on all these methods except the last one (Enumeration on the fly), because you can find many articles explaining the pros and cons for each one of them.
The destroying method
The array $ConcatenatedArrays is each time destroyed and replaced by the concatenation of both itself and the other array.
$ConcatenatedArrays += $Array1 $ConcatenatedArrays += $Array2 $ConcatenatedArrays += $Array3
The addition method
$ConcatenatedArrays = $Array1 + $Array2 + $Array3
Note:
Depending on the objects you handle, this method may give unexpected results, like getting an array of arrays instead of an array of all elements as single elements.
Enumeration
foreach($item in $Array1){$ConcatenatedArrays = $ConcatenatedArrays + $item} foreach($item in $Array2){$ConcatenatedArrays = $ConcatenatedArrays + $item} foreach($item in $Array3){$ConcatenatedArrays = $ConcatenatedArrays + $item}
Enumeration on the fly
Among all those know methods I recently proposed this one as answer on StackOverFlow.
$ConcatenatedArrays = $($Array1;$Array2;$Array3)
You need:
- Parentheses preceded by a dollar sign
- Elements inside the parentheses separated by semicolons (and NOT commas)
Technically, and without going too much in depth, this method executes the arrays inside the parentheses as code, by enumerating the values.
And, the $ sign before the parentheses indicates to PowerShell that the whole thing between the parentheses is to be considered as a value.
We can even list all arrays inside the parentheses without semicolons and use new lines instead.
$ConcatenatedArrays = $( $Array1 $Array2 $Array3 )
Why “on the fly”
This method can be used to build the global array at the same time you collect the data.
No need to do an extra operation to concatenate your data!
Let’s use the examples from the beginning of this article.
Example 1:
$AllFiles = $( Get-ChildItem -Path 'C:\Program Files\App1\Logs' Get-ChildItem -Path 'D:\App2\Logs' Get-ChildItem -Path 'E:\Logs' )
Example 2:
$AllUsers = $( Get-AdUser -Filter * -SearchBase 'OU=Management,DC=MyDomain,DC=Com' Get-AdUser -Filter * -SearchBase 'OU=InternalUsers,DC=MyDomain,DC=Com' Get-AdUser -Filter * -SearchBase 'OU=ExternalUsers,DC=MyDomain,DC=Com' )
Wait there is more!
This method is especially useful for one-liners.
Example 1:
$( Get-ChildItem -Path 'C:\Program Files\App1\Logs' Get-ChildItem -Path 'D:\App2\Logs' Get-ChildItem -Path 'E:\Logs' ) | ForEach-Object -Process { #Doing my stuff here on all collected files }
Example 2:
$( Get-AdUser -Filter * -SearchBase 'OU=Management,DC=MyDomain,DC=Com' Get-AdUser -Filter * -SearchBase 'OU=InternalUsers,DC=MyDomain,DC=Com' Get-AdUser -Filter * -SearchBase 'OU=ExternalUsers,DC=MyDomain,DC=Com' ) | ForEach-Object -Process { #Doing my stuff here on all collected user accounts }
And what about the speed?
Another important topic is to know if the concatenation on the fly is faster or slower than the other methods.
For this purpose, I will collect four lists of files and concatenate them into a single list.
You can unfold the line below to see the whole code:
$MaxIteration = 100 #region Destroying method 'Destroying method' Measure-Command -Expression { 1..$MaxIteration|%{ [array]$a = Get-ChildItem -path C:\ProgramData -File [array]$b = Get-ChildItem -path C:\Windows -File [array]$c = Get-ChildItem -path C:\Windows\System32 -File $ConcatenatedArray = $a $ConcatenatedArray += $b $ConcatenatedArray += $c $ConcatenatedArray | ForEach-Object -Process { $PSItem | Out-Null } } } #endregion Destroying method #region Addition method 'Addition method' Measure-Command -Expression { 1..$MaxIteration|%{ [array]$a = Get-ChildItem -path C:\ProgramData -File [array]$b = Get-ChildItem -path C:\Windows -File [array]$c = Get-ChildItem -path C:\Windows\System32 -File $ConcatenatedArray = $a + $b + $c $ConcatenatedArray | ForEach-Object -Process { $PSItem | Out-Null } } } #endregion Addition method #region Enumeration method 'Enumeration method' Measure-Command -Expression { 1..$MaxIteration|%{ [array]$a = Get-ChildItem -path C:\ProgramData -File [array]$b = Get-ChildItem -path C:\Windows -File [array]$c = Get-ChildItem -path C:\Windows\System32 -File [array]$ConcatenatedArray = $null foreach($i in $a){ $ConcatenatedArray = $ConcatenatedArray + $i } foreach($i in $b){ $ConcatenatedArray = $ConcatenatedArray + $i } foreach($i in $c){ $ConcatenatedArray = $ConcatenatedArray + $i } $ConcatenatedArray | ForEach-Object -Process { $PSItem | Out-Null } } } #endregion Enumeration method #region Concatenation on the fly 'Concatenation on the fly' Measure-Command -Expression { 1..$MaxIteration|%{ $( Get-ChildItem -path C:\ProgramData -File Get-ChildItem -path C:\Windows -File Get-ChildItem -path C:\Windows\System32 -File )| ForEach-Object -Process { $PSItem | Out-Null } } } #endregion Concatenation on the fly
And you can unfold the line below to see the result:
Destroying method Days : 0 Hours : 0 Minutes : 0 Seconds : 33 Milliseconds : 333 Ticks : 333337604 TotalDays : 0.000385807412037037 TotalHours : 0.00925937788888889 TotalMinutes : 0.555562673333333 TotalSeconds : 33.3337604 TotalMilliseconds : 33333.7604 Addition method Days : 0 Hours : 0 Minutes : 0 Seconds : 32 Milliseconds : 899 Ticks : 328990928 TotalDays : 0.000380776537037037 TotalHours : 0.00913863688888889 TotalMinutes : 0.548318213333333 TotalSeconds : 32.8990928 TotalMilliseconds : 32899.0928 Enumeration method Days : 0 Hours : 0 Minutes : 1 Seconds : 57 Milliseconds : 36 Ticks : 1170368437 TotalDays : 0.00135459309837963 TotalHours : 0.0325102343611111 TotalMinutes : 1.95061406166667 TotalSeconds : 117.0368437 TotalMilliseconds : 117036.8437 Concatenation on the fly Days : 0 Hours : 0 Minutes : 0 Seconds : 33 Milliseconds : 418 Ticks : 334184004 TotalDays : 0.000386787041666667 TotalHours : 0.009282889 TotalMinutes : 0.55697334 TotalSeconds : 33.4184004 TotalMilliseconds : 33418.4004
As you can notice, except the enumeration method, which is very slower, all other methods are equivalent in speed.
So it’s up to you to choose the method you feel the most comfortable with… :)