Minor tips for problems I was into in Jug, a parallel computing framework in Python.
I may be wrong, and if you know I misunderstand, please leave comments:
- Tasks can be created only outside functions.
Probably, this is common to create the main function with, if __name__ = __main__: main()
I did this, and called functions decorated with @TaskGenerator. However, jug could not launch tasks that are created in the main function. Instead, when I remove the main function, and write lines on the ground level (no indent level), the same code worked. - Multi-node parallelism can be achieved by just running "jug execute XXX.py" if the directory which stores the python code is shared by NFS. I could not find any clear comment on this, and I wondered how I can achieve multi-node parallelism.
- An atomic operation for File I/O can be achieved by creating a function decorated with @TaskGenerator. According to the summary of Jug, such a function is executed once. So, we can use it for an atomic operation.
Otherwise, I think Jug is excellent, and easy to use!
Still a question:
I do not know what jugfile can do.
On page17 of the documentation(https://jug.readthedocs.io/_/downloads/en/latest/pdf/), a post-process script import jugfile. I think the tasks in the main script make outputs to the jugfile, but do not know how to do.
On page17 of the documentation(https://jug.readthedocs.io/_/downloads/en/latest/pdf/), a post-process script import jugfile. I think the tasks in the main script make outputs to the jugfile, but do not know how to do.
