InThis article introduces ScaRLib, a Scala-based framework that aims to streamline the development cyber-physical swarms scenarios (i.e., systems of many interacting distributed devices that collectively accomplish system-wide tasks) by integrating macroprogramming and multi-agent reinforcement learning to design collective behavior. This framework serves as the starting point for a broader toolchain that will integrate these two approaches at multiple points to harness the capabilities of both, enabling the expression of complex and adaptive collective behavior.