You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* adding a mixture of spaces and tabs to unit tests
* adding parsing of arbitrary mpc constant assignment values
* adding parsing of arbitrary mpc matrixes
* adding ability to name matrix columns
* adding ability to extend matpower matrixes
* adding line splitter that is aware of matpower strings, extending data input tools to matlab cell arrays
* adding some basic documentation for data formats, adding notes to change log
Internally PowerModels utilizes a dictionary to store network data.
6
+
The dictionary uses strings as key values so this data structure can be serialized to JSON for algorithmic data exchange.
7
+
The data dictionary organization and key names are designed to be consistent with the [Matpower](http://www.pserc.cornell.edu/matpower/) file format and should be familiar to power system researchers.
8
+
9
+
The network data dictionary structure is roughly as follows,
10
+
```
11
+
{
12
+
"name":<string>,
13
+
"version":"2",
14
+
"baseMVA":<float>,
15
+
"bus":[
16
+
{
17
+
"index":<int>,
18
+
"bus_type":<int>,
19
+
"pd":<float>,
20
+
"qd":<float>,
21
+
...
22
+
},
23
+
...
24
+
],
25
+
"gen":[
26
+
{
27
+
"index":<int>,
28
+
"gen_bus":<int>,
29
+
"pg":<float>,
30
+
"qg":<float>,
31
+
...
32
+
},
33
+
...
34
+
],
35
+
"branch":[
36
+
{
37
+
"index":<int>,
38
+
"f_bus":<int>,
39
+
"t_bus":<int>,
40
+
"br_r":<int>,
41
+
...
42
+
},
43
+
...
44
+
],
45
+
"gencost":[
46
+
{
47
+
"index":<int>,
48
+
"model":<int>,
49
+
"startup":<float>,
50
+
"shutdown":<float>,
51
+
...
52
+
},
53
+
...
54
+
]
55
+
}
56
+
```
57
+
58
+
For a detailed list of all possible parameters please refer to the specification document provided with [Matpower](http://www.pserc.cornell.edu/matpower/). In addition to the traditional Matpower parameters every network component in the PowerModels dictionary has an `index` parameter, which can be used to uniquely identify that network element.
59
+
60
+
It is also important to note that although the Matpower format contains values in mixed units, during PowerModels data setup phase all of the data values are converted to per-unit and radian values.
61
+
62
+
63
+
## Matpower Data Files
64
+
65
+
The data exchange via JSON files is ideal for building algorithms, however it is hard to for humans to read and process.
66
+
To that end PowerModels also has extensive support for parsing Matpower network files in the `.m` format.
67
+
68
+
69
+
### User Extensions
70
+
71
+
In addition to parsing the standard Matpower parameters, PowerModels also supports extending the standard Matpower format in a number of ways as illustrated by the following examples. In these examples a JSON document fragments are used to indicate the structure of the PowerModel dictionary.
72
+
73
+
#### Single Values
74
+
Single values are added to the root of the dictionary as follows,
75
+
```
76
+
mpc.const_float = 4.56
77
+
```
78
+
becomes,
79
+
```
80
+
{
81
+
"const_float": 4.56
82
+
}
83
+
```
84
+
85
+
#### Nonstandard Matrices
86
+
Nonstandard matrices can be added as follows,
87
+
```
88
+
mpc.areas = [
89
+
1 1;
90
+
2 3;
91
+
];
92
+
```
93
+
becomes,
94
+
```
95
+
{
96
+
"areas":[
97
+
{
98
+
"index":1,
99
+
"col_1":1,
100
+
"col_2":1
101
+
},
102
+
{
103
+
"index":1,
104
+
"col_1":2,
105
+
"col_2":3
106
+
}
107
+
]
108
+
}
109
+
```
110
+
111
+
#### Column Names
112
+
Column names can be given to nonstandard matrices using the following special comment,
113
+
```
114
+
%column_names% area refbus
115
+
mpc.areas_named = [
116
+
4 5;
117
+
5 6;
118
+
];
119
+
```
120
+
becomes,
121
+
```
122
+
{
123
+
"areas":[
124
+
{
125
+
"index":1,
126
+
"area":4,
127
+
"refbus":5
128
+
},
129
+
{
130
+
"index":1,
131
+
"area":5,
132
+
"refbus":6
133
+
}
134
+
]
135
+
}
136
+
```
137
+
138
+
#### Standard Matrix Extentions
139
+
Finally, if a nonstandard matrix's name extends a current Matpower matrix name with an underscore, then its values will be merged with the original Matpower component data. Note that this feature requires that the nonstandard matrix has column names and has the same number of rows as the original matrix (similar to the `gencost` matrix in the Matpower format). For example,
# Attempts to determine the type of an array of strings extracted from a matlab file
199
+
functiontype_array(string_array)
200
+
value_string = [strip(value_string) for value_string in string_array]
201
+
202
+
ifany([contains(value_string, "'") for value_string in string_array])
203
+
value_array = [strip(value_string, '\'') for value_string in string_array]
204
+
else
205
+
ifany([contains(value_string, ".") ||contains(value_string, "e") for value_string in string_array])
206
+
value_array = [parse(Float64, value_string) for value_string in string_array]
207
+
else# otherwise assume it is an int
208
+
value_array = [parse(Int, value_string) for value_string in string_array]
209
+
end
210
+
end
211
+
212
+
return value_array
213
+
end
214
+
113
215
114
216
functionparse_matpower(file_string)
115
217
data_string =readstring(open(file_string))
@@ -187,6 +289,11 @@ function parse_matpower_data(data_string)
187
289
parsed_matrixes = []
188
290
parsed_cells = []
189
291
292
+
case =Dict{AbstractString,Any}(
293
+
"dcline"=> [],
294
+
"gencost"=> []
295
+
)
296
+
190
297
last_index =length(data_lines)
191
298
index =1
192
299
while index <= last_index
@@ -199,10 +306,13 @@ function parse_matpower_data(data_string)
199
306
200
307
ifcontains(line, "function mpc")
201
308
name =extract_assignment(line)
309
+
case["name"] = name
202
310
elseifcontains(line, "mpc.version")
203
-
version =extract_assignment(line)
311
+
version =extract_mpc_assignment(line)[2]
312
+
case["version"] = version
204
313
elseifcontains(line, "mpc.baseMVA")
205
-
baseMVA =parse(Float64, extract_assignment(line))
314
+
baseMVA =extract_mpc_assignment(line)[2]
315
+
case["baseMVA"] = baseMVA
206
316
elseifcontains(line, "[")
207
317
matrix =parse_matrix(data_lines, index)
208
318
push!(parsed_matrixes, matrix)
@@ -211,22 +321,33 @@ function parse_matpower_data(data_string)
211
321
cell =parse_cell(data_lines, index)
212
322
push!(parsed_cells, cell)
213
323
index = index + cell["line_count"]-1
324
+
elseifcontains(line, "mpc.")
325
+
name, value =extract_mpc_assignment(line)
326
+
case[name] = value
327
+
info("extending matpower format with value named: $(name)")
214
328
end
215
329
index +=1
216
330
end
217
331
218
-
case =Dict{AbstractString,Any}(
219
-
"name"=> name,
220
-
"version"=> version,
221
-
"baseMVA"=> baseMVA,
222
-
"dcline"=> [],
223
-
"gencost"=> []
224
-
)
332
+
if!haskey(case, "name")
333
+
warn(string("no case name found in matpower file. The file seems to be missing \"function mpc = ...\""))
334
+
case["name"] ="no_name_found"
335
+
end
336
+
337
+
if!haskey(case, "version")
338
+
warn(string("no case version found in matpower file. The file seems to be missing \"mpc.version = ...\""))
339
+
case["version"] ="unknown"
340
+
end
341
+
342
+
if!haskey(case, "baseMVA")
343
+
warn(string("no baseMVA found in matpower file. The file seems to be missing \"mpc.baseMVA = ...\""))
344
+
case["baseMVA"] =1.0
345
+
end
225
346
226
347
for parsed_matrix in parsed_matrixes
227
348
#println(parsed_matrix)
228
349
229
-
if parsed_matrix["name"] =="mpc.bus"
350
+
if parsed_matrix["name"] =="bus"
230
351
buses = []
231
352
232
353
for bus_row in parsed_matrix["data"]
@@ -259,7 +380,7 @@ function parse_matpower_data(data_string)
259
380
260
381
case["bus"] = buses
261
382
262
-
elseif parsed_matrix["name"] =="mpc.gen"
383
+
elseif parsed_matrix["name"] =="gen"
263
384
gens = []
264
385
265
386
for (i, gen_row) inenumerate(parsed_matrix["data"])
@@ -299,7 +420,7 @@ function parse_matpower_data(data_string)
299
420
300
421
case["gen"] = gens
301
422
302
-
elseif parsed_matrix["name"] =="mpc.branch"
423
+
elseif parsed_matrix["name"] =="branch"
303
424
branches = []
304
425
305
426
for (i, branch_row) inenumerate(parsed_matrix["data"])
@@ -335,7 +456,7 @@ function parse_matpower_data(data_string)
335
456
336
457
case["branch"] = branches
337
458
338
-
elseif parsed_matrix["name"] =="mpc.gencost"
459
+
elseif parsed_matrix["name"] =="gencost"
339
460
gencost = []
340
461
341
462
for (i, gencost_row) inenumerate(parsed_matrix["data"])
@@ -356,7 +477,7 @@ function parse_matpower_data(data_string)
356
477
error("incorrect Matpower file, the number of generator cost functions ($(length(case["gencost"]))) is inconsistent with the number of generators ($(length(case["gen"]))).\n")
357
478
end
358
479
359
-
elseif parsed_matrix["name"] =="mpc.dcline"
480
+
elseif parsed_matrix["name"] =="dcline"
360
481
dclines = []
361
482
362
483
for (i, dcline_row) inenumerate(parsed_matrix["data"])
@@ -392,35 +513,126 @@ function parse_matpower_data(data_string)
392
513
push!(dclines, dcline_data)
393
514
end
394
515
395
-
396
516
else
397
-
println(parsed_matrix["name"])
398
-
warn(string("unrecognized data matrix named \"", parsed_matrix["name"], "\" data was ignored."))
517
+
name = parsed_matrix["name"]
518
+
data = parsed_matrix["data"]
519
+
520
+
column_names = ["col_$(c)"for c in1:length(data[1])]
error("incorrect Matpower file, the number of bus names ($(length(parsed_cell["data"]))) is inconsistent with the number of buses ($(length(case["bus"]))).\n")
409
538
end
410
539
411
540
for (i, bus) inenumerate(case["bus"])
412
-
bus["bus_name"] = parsed_cell["data"][i]
541
+
# note striping the single quotes is not necessary in general, column typing takes care of this
ifany([startswith(name, "$(mp_name)_") for mp_name in matpower_matrix_names])
601
+
mp_name ="none"
602
+
mp_matrix ="none"
603
+
604
+
for mp_name in matpower_matrix_names
605
+
ifstartswith(name, "$(mp_name)_")
606
+
mp_matrix = case[mp_name]
607
+
break
608
+
end
609
+
end
610
+
611
+
if!has_column_names
612
+
error("failed to extend the matpower matrix \"$(mp_name)\" with the matrix \"$(name)\" because it does not have column names.")
613
+
end
614
+
615
+
iflength(mp_matrix) !=length(typed_dict_data)
616
+
error("failed to extend the matpower matrix \"$(mp_name)\" with the matrix \"$(name)\" because they do not have the same number of rows, $(length(mp_matrix)) and $(length(typed_dict_data)) respectively.")
617
+
end
618
+
619
+
info("extending matpower format by appending matrix \"$(name)\" onto \"$(mp_name)\"")
620
+
for (i, row) inenumerate(mp_matrix)
621
+
merge_row = typed_dict_data[i]
622
+
assert(row["index"] == merge_row["index"])
623
+
delete!(merge_row, "index")
624
+
for key inkeys(merge_row)
625
+
ifhaskey(row, key)
626
+
error("failed to extend the matpower matrix \"$(mp_name)\" with the matrix \"$(name)\" because they both share \"$(key)\" as a column name.")
627
+
end
628
+
row[key] = merge_row[key]
629
+
end
630
+
end
631
+
632
+
else
633
+
# minus 1 for excluding added "index" key
634
+
info("extending matpower format with data: $(name)$(length(typed_dict_data))x$(length(typed_dict_data[1])-1)")
0 commit comments