Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reimplement support for rknn detector #11365

Merged
merged 11 commits into from
May 21, 2024
Merged

Conversation

MarcA711
Copy link
Contributor

This PR is not complete. I just want to get some early feedback.

One thing that bothers me is that some parts (especially the prerequisites and setup parts) of the video processing and object detection docs overlap.
Compare https://github.com/MarcA711/frigate-rockchip/blob/1e1a11ae2bffab188e788e742c9ef0f801c7d002/docs/docs/configuration/hardware_acceleration.md?plain=1#L367-L397
and https://github.com/MarcA711/frigate-rockchip/blob/1e1a11ae2bffab188e788e742c9ef0f801c7d002/docs/docs/configuration/object_detectors.md?plain=1#L317-L346

Any suggestions how to handle this better?

Generally, it would be better for the user to describe the configuration for both video processing and object detection in one place. Now, the user has to find two pages. But I think this is again rockchip specific and difficult to fix without breaking other stuff.

Moreover, was it considered to move the post-processing part of all detectors in one file? As far as I know every detector implements its own post-processing for each supported model. However, post-processing of the same model (like yolo-nas, yolox etc.) should be the same so there is some duplicate code.

Copy link

netlify bot commented May 13, 2024

Deploy Preview for frigate-docs canceled.

Name Link
🔨 Latest commit 1837257
🔍 Latest deploy log https://app.netlify.com/sites/frigate-docs/deploys/664bace3166935000888643f

@NickM-27
Copy link
Sponsor Collaborator

If there are commonalities then you could put that in the installation section of the docs then have the hardware and object detector link to that as a prerequisite for specific setup

as far as post processing goes, it could make sense to have a util.py in detectors that handles this

@MarcA711
Copy link
Contributor Author

as far as post processing goes, it could make sense to have a util.py in detectors that handles this

I agree. I will try to implement post-processing for all detectors. I will use the mode: model_type: config option that is already implemented to handle this. But I will also include a way to return the pure detections (similar to how its already done) to allow the detectors to handle post-processing differently (e.g. using opencl on GPU).

@MarcA711
Copy link
Contributor Author

This is now ready to merge

@blakeblackshear
Copy link
Owner

I think it's best to add some information about the license on your repo where you published the models. Also, probably best to note that here as well. We wouldn't want someone using Frigate for commercial purposes to assume that these models are free for commercial use.

@blakeblackshear
Copy link
Owner

Lastly, how painful would it be to update this so the models are downloaded at runtime? That prevents frigate from actually redistributing the models itself.

@MarcA711
Copy link
Contributor Author

Good idea. I will add notes in my repo and frigate and I will exclude the models from the image. Maybe I will add some other models (maybe yolox) to the image later.

@MarcA711
Copy link
Contributor Author

I added a note to my repo, the docs and to the logger on each start, so that the user knows about the license situation.

I also excluded the models from the image.

@MarcA711
Copy link
Contributor Author

I see that Frigate 0.14 is coming out soon. Are there any plans for a beta phase? When should I finish my bigger changes so that they make it in that version?

@NickM-27
Copy link
Sponsor Collaborator

I don't know that I would call it "soon". There should be a beta within a week or two, and I am sure there will be at least a few betas to follow that.

@MarcA711
Copy link
Contributor Author

This PR is ready to merge from my side.

@blakeblackshear blakeblackshear merged commit e91f3d8 into blakeblackshear:dev May 21, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants