Symptoms

A one-to-many relationship record was created between 2 tables and Edit button was not appearing on the created custom related list.

Release

Any supported release.

The out of the box provided Edit... Global UI action should be visible on the on-to-many related list, but it was not appearing due to the reason that the RP.isOneToMany() condition on the "Edit..." UI Action fails.
As per the Configure the Edit option document, to make the one-to-many relationship with the related list, you need to set the "Reference float" option to true on dictionary level.
For one to many relationship, you don't have to create separate relationship record, the reference dictionary entry is enough to define the relationship.

Go to the reference dictionary entry which makes the relationship between the parent/child tables
Then Set the reference float option to true at the dictionary level as shown in the attached screenshot.