张 杰¹，夏 蕊¹，李 博¹，王学文¹，李娟莉¹，徐文军^1，2

（1. 太原理工大学 机械与运载工程学院，山西 太原 030002；2. 山西量界数字科技有限公司，山西 太原 030000）

引用格式：

张杰， 夏蕊， 李博， 等 . 基于长短期记忆-深度 Q 值网络的异构机器人煤矸协同分选方法［J］. 中国粉体技术， 2024， 30（3）：28-38.

ZHANG J， XIA R， LI B， et al. Heterogeneous Robot Coal Gangue Collaborative Sorting Method Based on long short term memory-deep Q network［J］. China Powder Science and Technology，2024，30（3）：28−38.

DOI：10.13732/j.issn.1008-5548.2024.03.003

收稿日期：2024-01-01，修回日期：2024-03-15，上线日期：2024-04-22。

基金项目：国家自然科学基金项目，编号 ：52204149；山西省自然科学基金项目，编号：202103021223080，202203021221051。

第一作者简介：张杰（1998—），男，硕士生，研究方向为智能煤矸分选。E-mail：zhangjie815921@163. com。

通信作者简介：李博（1979—），男，副教授，硕士生导师，三晋英才，研究方向为煤矸智能分选与控制。E-mail：libo@tyut. edu. cn。

摘要：【目的】提高传统的单一类别煤矸分选机器人在面对形状、尺寸差异较大的矸石时的适应性，分析异构机器人工 作特性，实现异构机器人协同分选。【方法】基于深度 Q值网络（deep Q network，DQN）提出异构机器人协同分选模型。 分析协同工作分选流程制定决策框架，根据强化学习所需，设计交互环境，构建智能体连续的状态空间奖惩函数，依据 长短期记忆网络（long short term memory， LTSM）和全连接网络相结合，构建DQN价值和目标网络，实现强化学习模型在 工作过程中的任务分配。【结果】协同分选模型与传统顺序分配模型相比，在不同含矸率工作负载下，可提高分选效 益 0. 49%~17. 74%；在样本含矸率为 21. 61%，传送带速度为 0. 4~0. 6 m/s 的条件下，可提高分选效率 2. 41%~8. 98%。 【结论】异构机器人协同分选方法可以在不同的工作负载下获得稳定的分拣效益，避免单一分配方案无法适应动态变化 的矸石流缺陷。

关键词：异构机器人；协同分选；强化学习；长短期记忆网络；深度Q值网络

Abstract

Objective Gangue is the waste and impurity produced during the process of coal mining and handling. Separating the coal gangue can reduce environmental pollution， improve energy efficiency， and provide economic benefits. Intelligent coal gangue sorting commonly employs robotic sorting and air-blowing separation. However， the use of a manipulator brings high costs， complexity， and failure rates. Additionally， a single air-blowing separation is not adaptable to gangue with significant differences in quality. By analysing the working characteristics of the two separation methods and designing a synergistic sorting system， the adaptability of the gangue sorting system can be improved， and the equipment cost can be reduced.

Methods This paper proposes the collaborative sorting hardware composition of heterogeneous robots. The paper combines deep reinforcement learning with the heterogeneous sorting robot of coal gangue， discretizing the continuous sorting process of coal gangue into a number of task segments. Overall planning is carried out for each task segment to give a feasible actuator cooperative work scheme. The third task set for gangue sorting and actuator collection is presented. To meet the continuity requirements of gangue sorting， we propose splitting the continuous task into several subsets. We allocate tasks using a buffer between the identification and sorting processes. Fourthly， this paper proposes a reinforcement learning decision-making framework based on LSTM-DQN （long short term memory， LTSM； deep Q network， DQN） to design the interaction environment of the model for reinforcement learning in the coal gangue sorting process. The framework includes the state space， action space， and reward function. Additionally， a cross-attention mechanism is used to compute the preference scores of different actuators for the task， which accelerates the convergence speed of the model. Fifthly， this paper constructs the core network of the model and introduces LSTM to handle state sequences with temporal and long-term dependencies. The DQN network structure is then optimized. Samples with different gangue rates are set up， and the proposed method and the sequential allocation model are compared in different gangue rates and different band speeds to reflect the superiority of the proposed method.

Results and Discussion Based on the proposed LTSM-DQN model， a method for sorting coal gangue using heterogeneous robots was developed. Six groups of samples with varying gangue rates were prepared to simulate different workloads. The experiment showed that the LTSM-DQN model is effective for task assignment in heterogeneous robot cooperation. Fig. 7 shows that various loads can be converged within 500 rounds of training. Samples with gangue rates ranging from 4. 73% to 30. 45% are sorted using the LTSM-DQN-based sorting model， which can limit the reduction of sorting efficiency to within 8%. When compared to the traditional sequential assignment， the sorting model based on LTSM-DQN can improve sorting efficiency by 2. 41% to 8. 98% under a gangue rate of 21. 61% and an adjusted belt speed of 0. 4~0. 6 m/s， as shown in Table 2. This improvement is significant and demonstrates the effectiveness of the LTSM-DQN model.

Conclusion A collaborative method for sorting heterogeneous robots and an optimised task allocation strategy using a reinforcement learning algorithm are proposed to achieve efficient and cost-effective sorting. The experiment demonstrates that this collaborative sorting method for coal gangue sorting can maintain the overall sorting benefit of the system at over 90% under different loads and is less affected by the belt speed compared to the traditional allocation square method， under different belt speeds and gangue content conditions. The cooperative sorting method is expected to evolve into the pneumatic sorting method and the multi-mechanic cooperative operation method. The system will be optimized in terms of multi-mechanic cooperation， air blowing， and robot cooperation. Reasonable and customized expansion will be carried out according to the actual needs of the mining area to satisfy specific sorting needs in a cost-effective manner.

Keywords：heterogeneous robots；cooperative sorting；reinforcement Learning；long short term memory；deep Q network

参考文献（References）

［1］FAN G W， ZHANG D S， WANG X F. Reduction and utilization of coal mine waste rock in China： a case study in Tiefa coalfield ［J］. Resources Conservation and Recycling，2014，83：24-33.

［2］YANG Y， ZENG Q， YIN G， et al. Vibration test of single coal gangue particle directly impacting the metal plate and the study of coal gangue recognition based on vibration signal and stacking integration ［J］. IEEE Access， 2019（7）： 106783- 106804.

［3］刘学雷. 我国选煤技术发展现状及趋势分析［J］. 选煤技术，2018（6）：12-15. LIU X L. Analysis of the current situation and trend of the development of coal beneficiation technology in China ［J］. Coal Selection Technology，2018（6）：12-15.

［4］MEYER E J， CRAIG I K. Dynamic model for a dense medium drum separator in coal beneficiation ［J］. Minerals Engineering，2015，77：78-85.

［5］AMBROS W M. Jigging： a review of fundamentals and future directions ［J］. Minerals，2020，10（11）：998-1029.

［6］ZHOU Y， ALBIJANIC B， TADESSE B， et al. Surface properties of aged coal and their effects on bubble particle attachment during flotation ［J］. Advanced Powder Technology，2020，31（4）：1490-1499.

［7］YANG Y， ZENG Q. Multipoint acceleration information acquisition of the impact experiments between coal gangue and the metal plate and coal gangue recognition based on SVM and serial splicing data ［J］. Arabian Journal for Science and Engineering，2021，46（3）：2749-2768. 

［8］LIU P， MA H W， ZHANG X H ， et al. On the equivalent position workspace for a coal gangue picking robot［C］// 2019 3rd International Conference on Artificial Intelligence， Auto-mation and Control Technologies. Xi'an：IOP science，2019：012078.

［9］SUN Z Y， HUANG L L， JIA R Q. Coal and gangue separating robot system based on computer vision ［J］. Sensors，2021， 21（4）：1349-1353.

［10］LIU P， TIAN H B， CAO X G， et al. Pick-and-place trajectory planning and robust adaptive fuzzy tracking control for cable based gangue sorting robots with model uncertainties and external disturbances ［J］. Machsines，2022，10（8）： 10080714.

［11］PENG L， XIN Z Q， XU H Z. Stability sensitivity for a cable-based coal-gangue picking robot based on grey relational analysis ［J］. International Journal of Advanced Robotic Systems，2021，18（6）：1059729.

［12］曹现刚，费佳浩，王鹏，等. 基于多机械手协同的煤矸分拣方法研究［J］. 煤炭科学技术，2019，47（4）：7-12. CAO X G， FEI J H， WANG P， et al. Research on coal gangue sorting method based on multi-mechanical arm collaboration ［J］. Coal Science and Technology，2019，47（4）：7-12.

［13］SHANG D， WANG Y， YANG Z， et al. Study on comprehensive calibration and image sieving for coal gangue separation parallel robot ［J］. Applied Sciences-Basel，2020，10（20）：10207059.

［14］WANG P， MA H， ZHANG Y， et al. A cooperative strategy of multi-arm coal gangue sorting robot based on immune dynamic workspace ［J］. International Journal of Coal Preparation and Utilization，2023，43（5）：794-814.

［15］MA H， WEI X， WANG P， et al. Multi-arm global cooperative coal gangue sorting method based on improved Hungarian algorithm ［J］. Sensors，2022，22（20）：22207987.

［16］WU X D， CAO X A， WANG P， et al. Multi-task allocation framework of coal gangue sorting robot system for the timevarying raw coal flow ［J］. International Journal of Coal Preparation and Utilization，2023：2217657

［17］张袁浩，潘祥生，陈晓晶，等. 智能选矸机器人关键技术研究［J］. 工矿自动化，2022，48（6）：69-76. ZHANG Y H， PAN X S， CHEN X J G， et al. Research on key technology of intelligent gangue selecting robot ［J］. Industrial and Mining Automation，2022，48（6）：69-76. 

［18］ZHENG K， DU C， LI J， et al. Underground pneumatic separation of coal and gangue with large size （≥50 mm） in green mining based on the machine vision system ［J］. Powder Technology，2015，278：223-233.

［19］ZHENG K， DU C， LI J， et al. Coal and gangue underground pneumatic separation effect evaluation influenced by different airflow directions ［J］. Advances in Materials Science and Engineering，2016：6465983.

［20］WANG Z X， XIE S X， CHEN G D， et al. An online flexible sorting model for coal and gangue based on multi-information fusion ［J］. IEEE Access，2021，9：90816-90827.

［21］VOLODYMYR， MNIH， KORAY， et al. Human-level control through deep reinforcement learning.［J］. Nature，2015， 518：529-533.