powershell hashtable key versus object property

This is understandable confusion, but as you said, hashtable keys are not object properties.

It can be tempting to treat them the same (and sometimes that works), but this is a situation where it definitely won't.

And part of the reason is that you're using a different cmdlet, not the direct language semantics.

Hashtables can use dot notation for their keys but the keys are not properties, and when you use the -Property parameter of Group-Object, you are looking for properties specifically, not just "anything you can access with a dot".

The alternative form of that parameter that takes a scriptblock, as you saw, is code that will be executed and so it's whatever value that block returns that will be grouped on.

To more directly answer your questions:

What is that second group-object actually doing?

It's looking for a property (specifically) on the current object (which is a hashtable).

If you want to see what the properties on one of those objects looks like, try this:

$list.Values[0].PSObject.Properties | ft

What does it think the 'order' property is?

It doesn't think it's anything; it looks for a property with that name and if it finds one it uses that value; otherwise it uses $null.

You'll get the same result with:

$list.Values | group -Property FakeProp

or

$list.Values | group -Property { $null }

Addressing your question in the comment:

Is there any way to know "when you should" and "when you shouldn't", Should I just defer to using script blocks, When is -Property usage preferred over { ... }?

-Property (without a scriptblock) is preferred whenever the value you want to group is available as a direct property of the object being inspected. If it's a property of a property, or some calculated value, or a hashtable key/value, or anything else, use a scriptblock. I'll call those "complex values".

If the complex value is useful, you may want to add it as an actual property of the object itself to encapsulate it; then you can reference it directly. This example isn't really appropriate for your hashtable situation but consider objects that represent people. They have 2 properties: Name and DateOfBirth.

$people is an array of these objects, and you want to group people by age (please ignore my inaccurate age-determining code).

$people | Group-Object -Property { ([DateTime]::Now - $_.DateOfBirth).Days / 365 -as [int] }

That's ok if you never need to know the age again; of course that's unlikely and also this looks a bit messy. It should be more immediately clear that you want to "group by age".

Instead, you can add your own (calculated) Age property to the existing objects with Add-Member:

$people |
    Add-Member -MemberType ScriptProperty -Name Age -Value {
        ([DateTime]::Now - $this.DateOfBirth).Days / 365 -as [int]
    } -Force

From here on out, each object in the $people array has an Age property that is calculated based on the value of the DateOfBirth property.

Now you can make your code clearer:

$people | Group-Object -Property Age

Again this doesn't really address your hashtable issue; the truth is they don't work that well for grouping. If you're going to do a lot of grouping with them, and you don't really need hashtables, make them into objects:

$objs = $list.Values |
    ForEach-Object -Process {
        New-Object -TypeName PSObject -Property $_ # takes a hashtable
    }

or

$objs = $list.Values |
    ForEach-Object -Process {
        [PSCustomObject]$_ # converts a hashtable to PSObject
    }

Then:

$objs | Group-Object -Property Order