| 引用本文: | 屈志坚,徐振清,周锐霖,朱丹.按列存储的配电网监测数据包区间编码正规化压缩处理[J].电力自动化设备,2018,(3): |
| QU Zhijian,XU Zhenqing,ZHOU Ruilin,ZHU Dan.Range encoding normalization compression processing for distribution network monitoring data pack with column storage[J].Electric Power Automation Equipment,2018,(3): |
|
| 摘要: |
| 针对不断增长的配电网按行存储数据的模式造成数据体量持续增大、压缩率难以提高的问题，利用按列存储的数据包架构和扩展区间编码的正规化处理技巧，提出了一种大体量配电网监测信息的压缩处理新方法。以列数据包为底层存储、知识网格为组织架构，将区间编码和正规化处理融入底层监测数据包的并行压缩处理进程中。以北京某动车段的配电网调度监测工程数据为算例，进行列数据包区间编码的正规化压缩测试。仿真结果表明：扩展区间编码的正规化处理技术可使监测信息压缩率提高至94%，获得了比已有云计算集群更好的压缩效果，通过动态调整映射区间，即可方便实现在无穷区间中对海量化监测信息编码压缩的工程应用需求。 |
| 关键词: 配电网 监测数据 列压缩 区间编码 正规化处理 数据处理 数据压缩 信息处理 |
| DOI：10.16081/j.issn.1006-6047.2018.03.008 |
| 分类号:TM761 |
| 基金项目:国家自然科学基金资助项目(51567008)；江西省杰出青年人才计划项目(20162BCB23045)；江西省自然科学基金资助项目(20161BAB206156,20171BAB206044)；江西省教育厅科技研究项目(GJJ160471) |
|
| Range encoding normalization compression processing for distribution network monitoring data pack with column storage |
|
QU Zhijian1, XU Zhenqing2, ZHOU Ruilin1, ZHU Dan1
|
|
1.School of Electrical Engineering, East China Jiaotong University, Nanchang 330013, China;2.CRRC Qingdao Sifang Co.,Ltd.,Qingdao 266000, China
|
| Abstract: |
| Aiming at the continually increasing large data sets and the difficulty of improving compression ratio caused by the increasing data storage mode by rows in distribution network, a new compression method for large distribution network monitoring data is proposed based on the data pack structure of column storage and the normalization processing of extended range encoding. Taking column data pack as underlying storage and knowledge grid as organization structure, the range encoding and normalization are integrated into the parallel compression process of the underlying monitoring data pack. Taking the distribution network monitoring data of Beijing train set depot section as an example, the column data pack range encoding normalization compression test is carried out. Simulative results show that the normalization of extension range encoding technology can increase the compression ratio of monitoring information to 94%,the compression effect is better than the existing cloud computing cluster, and the engineering application requirements of encoding the massive monitoring information in the infinite interval can be realized by dynamically adjusting the mapping range. |
| Key words: distribution network monitoring data column compression range encoding normalization data processing data compression information handling |
