Convert initializers to NumPower #356
Conversation
SkibidiProduction
requested review from
marclaporte,
mcharytoniuk and
andrewdalpino
SkibidiProduction
changed the title
There was a problem hiding this comment.
Hey @SkibidiProduction overall a pretty good start!
I really like the thoroughness of the unit tests but would encourage you to come up with some more descriptive names for the test methods instead of initializeTest1() initializeTest2() etc. I also like that you broke the initializers into Normal and Uniform versions. Finally, I like that you converted to more modern PHP syntax.
I'm concerned with strict types being unnecessary and slowing down our runtime. I'm also not keen on the idea of introducing additional exception classes without a compelling reason to do so. Also, I think we should stick to calling classes that provide API as "interfaces" and not introduce the "contract" term to mean the same thing as it has the potential to confuse people but provides no additional benefit. Lastly, I think we should avoid creating additional namespace hierarchies unless they are necessary.
src/NeuralNet/Initializers/Base/Contracts/AbstractInitializer.php
Outdated
Show resolved
Hide resolved
src/NeuralNet/Initializers/Base/Contracts/AbstractInitializer.php
Outdated
Show resolved
Hide resolved
src/NeuralNet/Initializers/Base/Contracts/AbstractInitializer.php
Outdated
Show resolved
Hide resolved
| @@ -0,0 +1,32 @@ | |||
| <?php | |||
There was a problem hiding this comment.
Let's colocate the interface with their implementations and avoid creating unnecessary/duplicate semantics around base classes and interfaces. The PHP language already provides ubiquitous semantics for these concepts.
There was a problem hiding this comment.
The Contracts namespace has been removed
| * @throws InvalidFanInException Initializer parameter fanIn is less than 1 | ||
| * @throws InvalidFanOutException Initializer parameter fanOut is less than 1 | ||
| */ | ||
| protected function validateInitParams(int $fanIn, int $fanOut) : void |
There was a problem hiding this comment.
This method name is necessarily generic for what it's actually doing. Since this method specifically validates the fan in and fan out perhaps a more accurate name would be validateFanInFanOut().
There was a problem hiding this comment.
the name has been changed
| } | ||
|
|
||
| #[Test] | ||
| #[TestDox('The initializer object os created correctly')] |
There was a problem hiding this comment.
Typo
#[TestDox('The initializer object is created correctly')]| #[Test] | ||
| #[TestDox('The result matrix has correct shape')] | ||
| #[DataProvider('initializeTest1DataProvider')] | ||
| public function initializeTest1(int $fanIn, int $fanOut) : void |
There was a problem hiding this comment.
Can we make these test method names more descriptive?
Example initializeWithCorrectShape()
There was a problem hiding this comment.
More meaningful names have been selected for the methods.
Once things are tested, would it make sense to propose changes as a PR at https://github.com/NumPower/numpower/pulls ? So these changes would have more visibility and perhaps lead to more collaboration throughout the ecosystem. |
@marclaporte Yes, but it's better not to change old classes, but to create new ones. Otherwise, tests will fail and this will complicate refactoring. |
Initializers have been converted to NumPower.
For He, Xavier, LeCun initializers, a division into two types was made. The first type is based on a truncated normal distribution, the second on a unified distribution.
The final distributions of the initializers are made in accordance with their analogs in the keras library.
A fork of the original NumPower is used (https://github.com/RubixML/numpower) due to the fact that there are difficulties with accepting changes and fixing bugs in the original extension.