libkylin-ai-system-private/ 0000775 0001750 0001750 00000000000 15160465216 014704 5 ustar feng feng libkylin-ai-system-private/.clang-format 0000775 0001750 0001750 00000000127 15160465216 017262 0 ustar feng feng
Language: Cpp
BasedOnStyle: Google
AccessModifierOffset: -4
IndentWidth: 4
TabWidth: 4 libkylin-ai-system-private/LICENSE 0000664 0001750 0001750 00000104513 15160465216 015715 0 ustar feng feng GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc.
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
Copyright (C)
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
Copyright (C)
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
.
libkylin-ai-system-private/CMakeLists.txt 0000664 0001750 0001750 00000000525 15160465216 017446 0 ustar feng feng cmake_minimum_required(VERSION 3.12)
project(libkylin-ai-system-private VERSION 1.0.0)
option(KYLIN_BUILD_AI_ENGINE "Build ai-engine library" ON)
option(KYLIN_BUILD_AI_PROTO "Build ai-proto definitions" ON)
if(KYLIN_BUILD_AI_ENGINE)
add_subdirectory(ai-engine)
endif()
if(KYLIN_BUILD_AI_PROTO)
add_subdirectory(ai-proto)
endif()
libkylin-ai-system-private/ai-engine/ 0000775 0001750 0001750 00000000000 15160465216 016540 5 ustar feng feng libkylin-ai-system-private/ai-engine/CMakeLists.txt 0000775 0001750 0001750 00000004447 15160465216 021314 0 ustar feng feng cmake_minimum_required(VERSION 3.12)
project(
"KylinAiEngine"
VERSION 0.0.1
DESCRIPTION
"A header-only c++ library that provides type safety and user defined literals for handling pyhsical values defined in the International System of Units."
HOMEPAGE_URL "https://github.com/bernedom/SI")
set(CMAKE_INCLUDE_CURRENT_DIR ON)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
include(GNUInstallDirs)
add_library(KylinAiEngine INTERFACE)
# Adding the install interface generator expression makes sure that the include
# files are installed to the proper location (provided by GNUInstallDirs)
target_include_directories(
KylinAiEngine
INTERFACE $
$)
target_compile_features(KylinAiEngine INTERFACE cxx_std_17)
# locations are provided by GNUInstallDirs
install(TARGETS KylinAiEngine
EXPORT KylinAiEngine_Targets
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
include(CMakePackageConfigHelpers)
write_basic_package_version_file("KylinAiEngineConfigVersion.cmake"
VERSION ${PROJECT_VERSION}
COMPATIBILITY SameMajorVersion)
configure_package_config_file(
"${PROJECT_SOURCE_DIR}/cmake/KylinAiEngineConfig.cmake.in"
"${PROJECT_BINARY_DIR}/KylinAiEngineConfig.cmake"
INSTALL_DESTINATION
${CMAKE_INSTALL_DATAROOTDIR}/${PROJECT_NAME}/cmake)
configure_file("${PROJECT_SOURCE_DIR}/cmake/KylinAiEngineFunctions.cmake"
${CMAKE_CURRENT_BINARY_DIR}/KylinAiEngineFunctions.cmake
COPYONLY)
install(EXPORT KylinAiEngine_Targets
FILE KylinAiEngineTargets.cmake
NAMESPACE KylinAiEngine::
DESTINATION ${CMAKE_INSTALL_DATAROOTDIR}/${PROJECT_NAME}/cmake)
install(FILES "${PROJECT_BINARY_DIR}/KylinAiEngineConfig.cmake"
"${PROJECT_BINARY_DIR}/KylinAiEngineConfigVersion.cmake"
"${CMAKE_CURRENT_BINARY_DIR}/KylinAiEngineFunctions.cmake"
DESTINATION ${CMAKE_INSTALL_DATAROOTDIR}/${PROJECT_NAME}/cmake)
install(DIRECTORY ${PROJECT_SOURCE_DIR}/include/kylin-ai DESTINATION include)
message(PROJECT_SOURCE_DIR::${PROJECT_SOURCE_DIR}::${CMAKE_INSTALL_INCLUDEDIR})
libkylin-ai-system-private/ai-engine/cmake/ 0000775 0001750 0001750 00000000000 15160465216 017620 5 ustar feng feng libkylin-ai-system-private/ai-engine/cmake/KylinAiEngineConfig.cmake.in 0000775 0001750 0001750 00000000277 15160465216 025054 0 ustar feng feng @PACKAGE_INIT@
include("${CMAKE_CURRENT_LIST_DIR}/@PROJECT_NAME@Targets.cmake")
include("${CMAKE_CURRENT_LIST_DIR}/@PROJECT_NAME@Functions.cmake")
check_required_components("@PROJECT_NAME@") libkylin-ai-system-private/ai-engine/cmake/KylinAiEngineFunctions.cmake 0000775 0001750 0001750 00000001102 15160465216 025176 0 ustar feng feng function (kylin_ai_add_engine_plugin PROJECT_NAME ENGINE_TYPE)
if (${ENGINE_TYPE} STREQUAL AI_ENGINE)
set(PLUGIN_PATH kylin-ai/plugins/ai-engines)
elseif(${ENGINE_TYPE} STREQUAL VECTORDB_ENGINE)
set(PLUGIN_PATH kylin-ai/plugins/db-engines/vector-database)
endif()
add_library(${PROJECT_NAME} SHARED ${ARGN})
target_link_libraries(${PROJECT_NAME} KylinAiEngine::KylinAiEngine)
include(GNUInstallDirs)
install(TARGETS ${PROJECT_NAME} DESTINATION
"${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}/${PLUGIN_PATH}")
endfunction() libkylin-ai-system-private/ai-engine/include/ 0000775 0001750 0001750 00000000000 15160465216 020163 5 ustar feng feng libkylin-ai-system-private/ai-engine/include/kylin-ai/ 0000775 0001750 0001750 00000000000 15160465216 021700 5 ustar feng feng libkylin-ai-system-private/ai-engine/include/kylin-ai/ai-engine/ 0000775 0001750 0001750 00000000000 15160465216 023534 5 ustar feng feng libkylin-ai-system-private/ai-engine/include/kylin-ai/ai-engine/textrecognitionengine.h 0000775 0001750 0001750 00000004034 15160465216 030324 0 ustar feng feng /*
* Copyright 2024 KylinSoft Co., Ltd.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see .
*/
#ifndef TEXTRECOGNITIONENGINE_H
#define TEXTRECOGNITIONENGINE_H
#include
#include
#include "aiengine.h"
#include "aiengineerror.h"
#include "aiengineresult.h"
namespace ai_engine {
namespace core_ai {
namespace vision {
typedef struct {
int x;
int y;
} PixelPoint;
struct TextLine {
std::string text;
std::vector points;
float conf;
};
struct TextRecognitionResult : result::AiEngineModelResult {
std::string allText;
std::vector lines;
};
using TextRecognitionResultCallback =
std::function;
class AbstractTextRecognitionEngine : public ai_engine::AbstractAiEngine {
public:
virtual ~AbstractTextRecognitionEngine() {}
virtual void setTextRecognitionResultCallback(
TextRecognitionResultCallback callback) = 0;
virtual result::AiEngineBaseResult init() = 0;
virtual result::AiEngineBaseResult destroy() = 0;
// 从图片文件中识别文本
virtual result::AiEngineBaseResult recognizeTextFromFile(
const std::string &imageFilePath) = 0;
// 从图片数据中识别文本
virtual result::AiEngineBaseResult recognizeTextFromBase64ImageData(
const std::string &imageBase64Data) = 0;
};
} // namespace vision
} // namespace core_ai
} // namespace ai_engine
#endif // TEXTRECOGNITIONENGINE_H libkylin-ai-system-private/ai-engine/include/kylin-ai/ai-engine/aiengineresult.h 0000775 0001750 0001750 00000001773 15160465216 026736 0 ustar feng feng /*
* Copyright 2024 KylinSoft Co., Ltd.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see .
*/
#ifndef AIENGINERESULT_H
#define AIENGINERESULT_H
#include
#include "aiengineerror.h"
namespace ai_engine {
namespace result {
struct AiEngineBaseResult {
bool success;
ai_engine::EngineError error;
};
struct AiEngineModelResult : public AiEngineBaseResult {
std::string modelName;
};
}
}
#endif libkylin-ai-system-private/ai-engine/include/kylin-ai/ai-engine/textgenerationengine.h 0000775 0001750 0001750 00000003666 15160465216 030151 0 ustar feng feng /*
* Copyright 2024 KylinSoft Co., Ltd.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see .
*/
#ifndef GENAITEXTENGINE_H
#define GENAITEXTENGINE_H
#include
#include
#include "aiengine.h"
#include "aiengineerror.h"
#include "aiengineresult.h"
namespace ai_engine {
namespace gen_ai {
namespace nlp {
struct TextGenerationResult : result::AiEngineModelResult {
std::string text;
};
using TextGenerationResultCallback =
std::function;
class AbstractTextGenerationEngine : public ai_engine::AbstractAiEngine {
public:
virtual ~AbstractTextGenerationEngine() {}
virtual void setTextGenerationResultCallback(
TextGenerationResultCallback callback) = 0;
// 初始化对话模块
virtual result::AiEngineBaseResult init() = 0;
// 销毁对话模块
virtual result::AiEngineBaseResult destroy() = 0;
// 生成内容
virtual result::AiEngineBaseResult generateContent(
const std::string &input) = 0;
// 对话
// message 为json格式的字符串
virtual result::AiEngineBaseResult chat(const std::string &message) = 0;
// 停止生成
virtual void stopGenerateContent() = 0;
// 停止对话
virtual void stopChat() = 0;
};
} // namespace nlp
} // namespace gen_ai
} // namespace ai_engine
#endif // GENAITEXTENGINE_H
libkylin-ai-system-private/ai-engine/include/kylin-ai/ai-engine/aienginepluginfactory.h 0000775 0001750 0001750 00000004477 15160465216 030312 0 ustar feng feng /*
* Copyright 2024 KylinSoft Co., Ltd.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see .
*/
#ifndef AIENGINEPLUGINFACTORY_H
#define AIENGINEPLUGINFACTORY_H
#include
#include "embeddingengine.h"
#include "imagegenerationengine.h"
#include "imageprocessengine.h"
#include "speechengine.h"
#include "textgenerationengine.h"
#include "textrecognitionengine.h"
namespace ai_engine {
class AbstractAiEnginePluginFactory {
public:
virtual ~AbstractAiEnginePluginFactory() {}
virtual std::string name() const = 0;
virtual std::unique_ptr
createTextGenerationEngine() {
return nullptr;
}
virtual std::unique_ptr
createImageGenerationEngine() {
return nullptr;
}
virtual std::unique_ptr
createSpeechEngine() {
return nullptr;
}
virtual std::unique_ptr
createTextRecognitionEngine() {
return nullptr;
}
virtual std::unique_ptr
createImageProcessEngine() {
return nullptr;
}
virtual std::unique_ptr
createEmbeddingEngine() {
return nullptr;
}
};
} // namespace ai_engine
#define AI_ENGINE_PLUGIN_FACTORY(ClassName) \
extern "C" { \
::ai_engine::AbstractAiEnginePluginFactory* \
ai_engine_plugin_factory_instance() { \
static ClassName factory; \
return &factory; \
} \
}
#endif // AIENGINEPLUGINFACTORY_H libkylin-ai-system-private/ai-engine/include/kylin-ai/ai-engine/speechengine.h 0000775 0001750 0001750 00000006750 15160465216 026355 0 ustar feng feng /*
* Copyright 2024 KylinSoft Co., Ltd.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see .
*/
#ifndef COREAISPEECHENGINE_H
#define COREAISPEECHENGINE_H
#include
#include
#include
#include
#include "aiengine.h"
#include "aiengineerror.h"
#include "aiengineresult.h"
namespace ai_engine {
namespace core_ai {
namespace speech {
enum class SpeechResultReason {
// 出错
SpeechErrorOccurred = 1,
// 已启动语音识别
SpeechRecognitionStarted = 2,
// 正在进行语音识别，中间结果
SpeechRecognizing = 3,
// 语音识别的最终结果，经过修正
SpeechRecognized = 4,
// 语音识别完成
SpeechRecognitionCompleted = 5,
// 已启动语音合成
SpeechSynthesisStarted = 6,
// 正在进行语音合成，语音合成的结果输出
SpeechSynthesizing = 7,
// 语音合成已经完成
SpeechSynthesisCompleted = 8,
};
struct RecognitionResult : result::AiEngineModelResult {
std::string text;
int speakerId;
SpeechResultReason reason;
};
struct SynthesisResult : result::AiEngineModelResult {
std::string audioFormat;
int audioRate;
int audioChannel;
std::string base64AudioData;
SpeechResultReason reason;
};
using RecognitionResultCallback =
std::function;
using SynthesisResultCallback =
std::function;
class AbstractSpeechEngine : public ai_engine::AbstractAiEngine {
public:
virtual ~AbstractSpeechEngine() {}
virtual void setRecognitionResultCallback(
RecognitionResultCallback callback) = 0;
virtual void setSynthesisResultCallback(
SynthesisResultCallback callback) = 0;
///[0] 流式语音识别
virtual result::AiEngineBaseResult initContinuousRecognitionModule() = 0;
virtual result::AiEngineBaseResult destroyContinuousRecognitionModule() = 0;
virtual result::AiEngineBaseResult startContinuousRecognition(
const std::string ¶ms) = 0;
virtual result::AiEngineBaseResult writeContinuousRecognitionAudioData(
const std::vector &audioData) = 0;
virtual result::AiEngineBaseResult stopContinuousRecognition() = 0;
///[0]
///[1] 非流式语音识别
virtual result::AiEngineBaseResult initRecognizeOnceModule() = 0;
virtual result::AiEngineBaseResult destroyRecognizeOnceModule() = 0;
virtual result::AiEngineBaseResult recognizeOnce(
const std::string ¶ms, const std::vector &audioData) = 0;
///[1]
///[2] 语音合成
virtual result::AiEngineBaseResult initSynthesizeModule() = 0;
virtual result::AiEngineBaseResult destroySynthesizeModule() = 0;
virtual result::AiEngineBaseResult synthesizeText(
const std::string ¶ms, const std::string &text) = 0;
///[2]
};
} // namespace speech
} // namespace core_ai
} // namespace ai_engine
#endif libkylin-ai-system-private/ai-engine/include/kylin-ai/ai-engine/imageprocessengine.h 0000775 0001750 0001750 00000004507 15160465216 027565 0 ustar feng feng /*
* Copyright 2024 KylinSoft Co., Ltd.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see .
*/
#ifndef COREAIIMAGEPROCESSENGINE_H
#define COREAIIMAGEPROCESSENGINE_H
#include
#include
#include "aiengine.h"
#include "aiengineerror.h"
#include "aiengineresult.h"
namespace ai_engine {
namespace core_ai {
namespace vision {
struct ImageResult : public result::AiEngineModelResult {
std::string imageFormat;
std::string imageBase64Data;
};
using ImageResultCallback = std::function;
class AbstractImageProcessEngine : public AbstractAiEngine {
public:
virtual ~AbstractImageProcessEngine() {}
// 抠图模块
virtual result::AiEngineBaseResult initMattingsModule() = 0;
virtual result::AiEngineBaseResult destroyMattingsModule() = 0;
virtual void setMattingResultCallback(ImageResultCallback callback) = 0;
virtual result::AiEngineBaseResult mattingFromBase64ImageData(
const std::string &imageBase64Data) = 0;
virtual result::AiEngineBaseResult mattingFromImageFile(
const std::string &imageFilePath) = 0;
virtual void stopMattingProcess() = 0;
// 图像二值化模块
virtual result::AiEngineBaseResult initImageBinaryModule() = 0;
virtual result::AiEngineBaseResult destroyImageBinaryModule() = 0;
virtual void setImageBinaryResultCallback(ImageResultCallback callback) = 0;
virtual result::AiEngineBaseResult binarizeImageFromBase64ImageData(
const std::string &imageBase64Data) = 0;
virtual result::AiEngineBaseResult binarizeImageFromImageFile(
const std::string &imageFilePath) = 0;
virtual void stopImageBinarization() = 0;
};
} // namespace vision
} // namespace core_ai
} // namespace ai_engine
#endif libkylin-ai-system-private/ai-engine/include/kylin-ai/ai-engine/embeddingengine.h 0000775 0001750 0001750 00000004302 15160465216 027013 0 ustar feng feng /*
* Copyright 2024 KylinSoft Co., Ltd.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see .
*/
#ifndef EMBEDDINGENGINE_H
#define EMBEDDINGENGINE_H
#include
#include
#include
#include "aiengine.h"
#include "aiengineerror.h"
#include "aiengineresult.h"
namespace ai_engine {
namespace core_ai {
namespace embedding {
struct EmbeddingResult : result::AiEngineModelResult {
std::vector embeddings;
};
using EmbeddingResultCallback =
std::function;
class AbstractEmbeddingEngine : public ai_engine::AbstractAiEngine {
public:
virtual ~AbstractEmbeddingEngine() {}
virtual void setEmbeddingResultCallback(
EmbeddingResultCallback callback) = 0;
///[0] 文本向量化
virtual result::AiEngineBaseResult initTextEmbeddingModule() = 0;
virtual result::AiEngineBaseResult destroyTextEmbeddingModule() = 0;
virtual result::AiEngineBaseResult embeddingText(
const std::string &text) = 0;
///[0]
///[1] 图像向量化
virtual result::AiEngineBaseResult initImageEmbeddingModule() = 0;
virtual result::AiEngineBaseResult destroyImageEmbeddingModule() = 0;
virtual result::AiEngineBaseResult embeddingImageFromImageFile(
const std::string &filePath) = 0;
virtual result::AiEngineBaseResult embeddingImageFromBase64ImageData(
const std::string &fileData) = 0;
virtual result::AiEngineBaseResult embeddingTextWithImageModel(
const std::string &text) = 0;
///[1]
};
} // namespace embedding
} // namespace core_ai
} // namespace ai_engine
#endif // EMBEDDINGENGINE_H
libkylin-ai-system-private/ai-engine/include/kylin-ai/ai-engine/aiengineerror.h 0000775 0001750 0001750 00000006342 15160465216 026546 0 ustar feng feng /*
* Copyright 2024 KylinSoft Co., Ltd.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see .
*/
#ifndef AIENGINEERROR_H
#define AIENGINEERROR_H
#include
namespace ai_engine {
// 引擎相关错误分类
enum class EngineErrorCategory {
Invalid = -1,
Initialization,
InputError,
OutputError
};
#define PLUGIN_COMMON_ERROR_CODE \
/*通用初始化相关错误*/ \
NomoreMemory, \
\
/*本地模型初始化相关错误*/ \
FileNotFound, VersionMismatched, FilePermissionDenied, \
\
/*云端模型初始化相关错误*/ \
NetworkDisconnected, NetworkLatencyTooLong, FailedToConnectServer, \
ServerNoResponse, NoServerService, SslCertificateError, Unauthorized, \
\
/*输入相关错误*/ \
TooLittleParams, TooManyParams, InvalidParamType, TooLittleData, \
TooManyData, TooFrequentRequest, TooManyRequest, UnknownError = 1000,
// NLP引擎相关错误码
enum class NlpEngineErrorCode {
PLUGIN_COMMON_ERROR_CODE
// 输出相关错误
FailedToAnswer,
InvalidWords,
InputTextBlocked,
OutputTextBlocked,
FreeAllocatedQuotaExceeded,
LocalNlpModelNotLoaded,
};
// 语音引擎相关错误码
enum class SpeechEngineErrorCode {
PLUGIN_COMMON_ERROR_CODE
// 输出相关错误
UnsupportedLanguage,
UnsupportedScript,
InvalidWakeupWords
};
// 图像引擎相关错误码
enum class VisionEngineErrorCode {
PLUGIN_COMMON_ERROR_CODE
// 输出相关错误
Invalidwords,
ImageGenerationFailed,
ImageCensorshipFailed,
OcrModelError,
ImageFileReadError,
ModelEarlyLoaded,
OcrTimeout,
MattingModelError,
DdcpModelError,
ImageNumberInvalid,
ImageSizeInvalid
};
class EngineError {
public:
EngineError() = default;
EngineError(int code, const std::string& message)
: code_(code), message_(message) {}
const std::string& getModule() const { return module_; }
int getCode() const { return code_; }
const std::string& getMessage() const { return message_; }
private:
std::string module_{"AI Engine"};
int code_{-1};
std::string message_;
};
} // namespace ai_engine
#endif
libkylin-ai-system-private/ai-engine/include/kylin-ai/ai-engine/aiengine.h 0000775 0001750 0001750 00000004574 15160465216 025501 0 ustar feng feng /*
* Copyright 2024 KylinSoft Co., Ltd.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see .
*/
#ifndef AIENGINE_LM_H
#define AIENGINE_LM_H
#include
namespace ai_engine {
class AbstractAiEngine {
public:
virtual ~AbstractAiEngine() {}
// 引擎的名称
virtual std::string engineName() const = 0;
// 是否是云端引擎
virtual bool isCloud() const = 0;
// 是否是内置引擎
// AI子系统提供内置的模型，调用者不可以指定和切换模型
// 包括向量化模型、OCR等
virtual bool isBuiltInEngine() const = 0;
// 是否是自定义模型引擎
// 自定义模型引擎需要满足openai chat api接口格式
virtual bool isCustomModelEngine() const { return false; }
// 引擎使用的模型信息
// json格式的字符串，包括模型的名称，厂商，需要的配置信息等
// 向量化
virtual std::string modelInfo() const = 0;
// 设置当前引擎使用的模型
// 针对使用一个模型实现所有功能的引擎，如果是内置引擎，则该接口无效
virtual bool setCurrentModel(const std::string &model) {
return false;
}
// 当前引擎使用的模型
// 针对使用一个模型实现所有功能的引擎，返回的是modelVersion
virtual std::string currentModel() const { return {}; }
// 当前引擎使用的模型名称
virtual std::string currentModelName() const { return {}; }
// 设置模型的配置信息
// 1. json格式的字符串，部分字段需要从modelInfo中获取
// 2. 云端模型可能需要配置账号信息
// 3. 自定义模型需要配置模型的请求地址、apiKey和模型模型等
virtual void setConfig(const std::string &config) {};
};
} // namespace ai_engine
#endif // AIENGINE_H
libkylin-ai-system-private/ai-engine/include/kylin-ai/ai-engine/imagegenerationengine.h 0000775 0001750 0001750 00000004662 15160465216 030244 0 ustar feng feng /*
* Copyright 2024 KylinSoft Co., Ltd.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see .
*/
#ifndef GENAIIMAGEENGINE_H
#define GENAIIMAGEENGINE_H
#include
#include
#include
#include
#include "aiengine.h"
#include "aiengineerror.h"
#include "aiengineresult.h"
namespace ai_engine {
namespace gen_ai {
namespace vision {
struct ImageGenerationResult : result::AiEngineModelResult {
int width;
int height;
int index;
int total;
std::string format;
std::string imageBase64Data;
};
using ImageGenerationCallback =
std::function;
class AbstractImageGenerationEngine : public ai_engine::AbstractAiEngine {
public:
virtual ~AbstractImageGenerationEngine() {}
virtual void setGenerationResultCallback(
ImageGenerationCallback callback) = 0;
// 初始化图像生成模块
virtual result::AiEngineBaseResult init() = 0;
// 销毁图像生成模块
virtual result::AiEngineBaseResult destroy() = 0;
// 获取支持的图像生成参数
virtual std::vector supportedImageStyles() const = 0;
// 获取支持的图像生成数量
virtual int supportedImageNumber() const = 0;
// 获取支持的图像生成尺寸
virtual std::vector> supportedImageSizes() const = 0;
// 设置图像生成风格
virtual void setImageStyle(const std::string &style) = 0;
// 设置图像生成数量
virtual void setImageNumber(int number) = 0;
// 设置图像生成尺寸
virtual void setImageSize(int width, int height) = 0;
// 生成图像
virtual result::AiEngineBaseResult generateImage(
const std::string &prompt) = 0;
virtual void stopImageGeneration() = 0;
};
} // namespace vision
} // namespace gen_ai
} // namespace ai_engine
#endif libkylin-ai-system-private/ai-engine/include/kylin-ai/aiengine.h 0000775 0001750 0001750 00000002042 15160465216 023631 0 ustar feng feng /*
* Copyright 2024 KylinSoft Co., Ltd.
*
* This program is free software: you can redistribute it and/or modify it under
* the terms of the GNU General Public License as published by the Free Software
* Foundation, either version 3 of the License, or (at your option) any later
* version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program. If not, see .
*/
#include "ai-engine/aiengine.h"
#include "ai-engine/aiengineerror.h"
#include "ai-engine/aienginepluginfactory.h"
#include "ai-engine/embeddingengine.h"
#include "ai-engine/speechengine.h"
#include "ai-engine/textrecognitionengine.h"
#include "ai-engine/imageprocessengine.h"
#include "ai-engine/imagegenerationengine.h"
#include "ai-engine/textgenerationengine.h" libkylin-ai-system-private/ai-proto/ 0000775 0001750 0001750 00000000000 15160465216 016436 5 ustar feng feng libkylin-ai-system-private/ai-proto/protocols/ 0000775 0001750 0001750 00000000000 15160465216 020462 5 ustar feng feng libkylin-ai-system-private/ai-proto/protocols/usr/ 0000775 0001750 0001750 00000000000 15160465216 021273 5 ustar feng feng libkylin-ai-system-private/ai-proto/protocols/usr/share/ 0000775 0001750 0001750 00000000000 15160465216 022375 5 ustar feng feng libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/ 0000775 0001750 0001750 00000000000 15160465216 024112 5 ustar feng feng libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/ 0000775 0001750 0001750 00000000000 15160465216 026136 5 ustar feng feng libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/ 0000775 0001750 0001750 00000000000 15160465216 027242 5 ustar feng feng libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/gennlpservice.xml 0000664 0001750 0001750 00000003035 15160465216 032631 0 ustar feng feng
././@LongLink 0000644 0000000 0000000 00000000157 00000000000 011606 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/coreimageembeddingservice.xml libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/coreimageembeddings0000664 0001750 0001750 00000002357 15160465216 033151 0 ustar feng feng
././@LongLink 0000644 0000000 0000000 00000000146 00000000000 011604 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/assistantservice.xml libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/assistantservice.xm0000664 0001750 0001750 00000005563 15160465216 033213 0 ustar feng feng
libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/knowledgebase.xml 0000664 0001750 0001750 00000013161 15160465216 032600 0 ustar feng feng
././@LongLink 0000644 0000000 0000000 00000000153 00000000000 011602 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/datamanagementservice.xml libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/datamanagementservi0000664 0001750 0001750 00000006040 15160465216 033204 0 ustar feng feng
././@LongLink 0000644 0000000 0000000 00000000150 00000000000 011577 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/genaivisionservice.xml libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/genaivisionservice.0000664 0001750 0001750 00000002726 15160465216 033146 0 ustar feng feng
././@LongLink 0000644 0000000 0000000 00000000147 00000000000 011605 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/corevisionservice.xml libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/corevisionservice.x0000664 0001750 0001750 00000002277 15160465216 033204 0 ustar feng feng
libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/configservice.xml 0000664 0001750 0001750 00000007407 15160465216 032622 0 ustar feng feng
././@LongLink 0000644 0000000 0000000 00000000147 00000000000 011605 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/corespeechservice.xml libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/corespeechservice.x0000664 0001750 0001750 00000005066 15160465216 033143 0 ustar feng feng
././@LongLink 0000644 0000000 0000000 00000000156 00000000000 011605 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/coretextembeddingservice.xml libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/gdbus/coretextembeddingse0000664 0001750 0001750 00000001562 15160465216 033215 0 ustar feng feng
libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/ 0000775 0001750 0001750 00000000000 15160465216 027071 5 ustar feng feng libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/ 0000775 0001750 0001750 00000000000 15160465216 030756 5 ustar feng feng ././@LongLink 0000644 0000000 0000000 00000000147 00000000000 011605 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/schema.proto libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/schema.pro0000664 0001750 0001750 00000011144 15160465216 032741 0 ustar feng feng syntax = "proto3";
package milvus.proto.schema;
option go_package = "github.com/milvus-io/milvus-proto/go-api/v2/schemapb";
option java_multiple_files = true;
option java_package = "io.milvus.grpc";
option java_outer_classname = "SchemaProto";
option java_generate_equals_and_hash = true;
option csharp_namespace = "Milvus.Client.Grpc";
import "common.proto";
import "google/protobuf/descriptor.proto";
/**
* @brief Field data type
*/
enum DataType {
None = 0;
Bool = 1;
Int8 = 2;
Int16 = 3;
Int32 = 4;
Int64 = 5;
Float = 10;
Double = 11;
String = 20;
VarChar = 21; // variable-length strings with a specified maximum length
Array = 22;
JSON = 23;
BinaryVector = 100;
FloatVector = 101;
Float16Vector = 102;
BFloat16Vector = 103;
SparseFloatVector = 104;
}
enum FieldState {
FieldCreated = 0;
FieldCreating = 1;
FieldDropping = 2;
FieldDropped = 3;
}
/**
* @brief Field schema
*/
message FieldSchema {
int64 fieldID = 1;
string name = 2;
bool is_primary_key = 3;
string description = 4;
DataType data_type = 5;
repeated common.KeyValuePair type_params = 6;
repeated common.KeyValuePair index_params = 7;
bool autoID = 8;
FieldState state = 9; // To keep compatible with older version, the default
// state is `Created`.
DataType element_type = 10; // For array type, the element type is stored here
ValueField default_value = 11; // default_value only support scalars except array and json for now
bool is_dynamic = 12; // mark whether this field is the dynamic field
bool is_partition_key = 13; // enable logic partitions
bool is_clustering_key = 14;
}
/**
* @brief Collection schema
*/
message CollectionSchema {
string name = 1;
string description = 2;
bool autoID = 3 [deprecated=true]; // deprecated later, keep compatible with c++ part now
repeated FieldSchema fields = 4;
bool enable_dynamic_field = 5; // mark whether this table has the dynamic field function enabled.
repeated common.KeyValuePair properties = 6;
}
message BoolArray { repeated bool data = 1; }
message IntArray { repeated int32 data = 1; }
message LongArray { repeated int64 data = 1; }
message FloatArray { repeated float data = 1; }
message DoubleArray { repeated double data = 1; }
// For special fields such as bigdecimal, array...
message BytesArray { repeated bytes data = 1; }
message StringArray { repeated string data = 1; }
message ArrayArray {
repeated ScalarField data = 1;
DataType element_type = 2;
}
message JSONArray { repeated bytes data = 1; }
message ValueField {
oneof data {
bool bool_data = 1;
int32 int_data = 2;
int64 long_data = 3;
float float_data = 4;
double double_data = 5;
string string_data = 6;
bytes bytes_data = 7;
}
}
message ScalarField {
oneof data {
BoolArray bool_data = 1;
IntArray int_data = 2;
LongArray long_data = 3;
FloatArray float_data = 4;
DoubleArray double_data = 5;
StringArray string_data = 6;
BytesArray bytes_data = 7;
ArrayArray array_data = 8;
JSONArray json_data = 9;
}
}
// beta, api may change
message SparseFloatArray {
repeated bytes contents = 1;
// dim is the max dimension of the current batch of vectors
int64 dim = 2;
}
message VectorField {
// For sparse vector, dim is the max dimension of the current batch of vectors
int64 dim = 1;
oneof data {
FloatArray float_vector = 2;
bytes binary_vector = 3;
bytes float16_vector = 4;
bytes bfloat16_vector = 5;
SparseFloatArray sparse_float_vector = 6;
}
}
message FieldData {
DataType type = 1;
string field_name = 2;
oneof field {
ScalarField scalars = 3;
VectorField vectors = 4;
}
int64 field_id = 5;
bool is_dynamic = 6;
}
message IDs {
oneof id_field {
LongArray int_id = 1;
StringArray str_id = 2;
}
}
message SearchResultData {
int64 num_queries = 1;
int64 top_k = 2;
repeated FieldData fields_data = 3;
repeated float scores = 4;
IDs ids = 5;
repeated int64 topks = 6;
repeated string output_fields = 7;
FieldData group_by_field_value = 8;
int64 all_search_count = 9;
repeated float distances = 10;
}
// vector field clustering info
message VectorClusteringInfo {
// for multi vectors
string field = 1;
schema.VectorField centroid = 2;
}
// Scalar field clustering info
// todo more definitions: min/max, etc
message ScalarClusteringInfo {
string field = 1;
}
// clustering distribution info of a certain data unit, it can be segment, partition, etc.
message ClusteringInfo {
repeated VectorClusteringInfo vector_clustering_infos = 1;
repeated ScalarClusteringInfo scalar_clustering_infos = 2;
}
libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/msg.proto 0000664 0001750 0001750 00000004726 15160465216 032642 0 ustar feng feng syntax = "proto3";
package milvus.proto.msg;
option go_package = "github.com/milvus-io/milvus-proto/go-api/v2/msgpb";
import "common.proto";
import "schema.proto";
enum InsertDataVersion {
// 0 must refer to row-based format, since it's the first version in Milvus.
RowBased = 0;
ColumnBased = 1;
}
message InsertRequest {
common.MsgBase base = 1;
string shardName = 2;
string db_name = 3;
string collection_name = 4;
string partition_name = 5;
int64 dbID = 6;
int64 collectionID = 7;
int64 partitionID = 8;
int64 segmentID = 9;
repeated uint64 timestamps = 10;
repeated int64 rowIDs = 11;
// row_data was reserved for compatibility
repeated common.Blob row_data = 12;
repeated schema.FieldData fields_data = 13;
uint64 num_rows = 14;
InsertDataVersion version = 15;
}
message DeleteRequest {
common.MsgBase base = 1;
string shardName = 2;
string db_name = 3;
string collection_name = 4;
string partition_name = 5;
int64 dbID = 6;
int64 collectionID = 7;
int64 partitionID = 8;
repeated int64 int64_primary_keys = 9; // deprecated
repeated uint64 timestamps = 10;
int64 num_rows = 11;
schema.IDs primary_keys = 12;
}
message MsgPosition {
string channel_name = 1;
bytes msgID = 2;
string msgGroup = 3;
uint64 timestamp = 4;
}
message CreateCollectionRequest {
common.MsgBase base = 1;
string db_name = 2;
string collectionName = 3;
string partitionName = 4;
// `schema` is the serialized `schema.CollectionSchema`
int64 dbID = 5;
int64 collectionID = 6;
int64 partitionID = 7; // deprecated
bytes schema = 8;
repeated string virtualChannelNames = 9;
repeated string physicalChannelNames = 10;
repeated int64 partitionIDs = 11;
}
message DropCollectionRequest {
common.MsgBase base = 1;
string db_name = 2;
string collectionName = 3;
int64 dbID = 4;
int64 collectionID = 5;
}
message CreatePartitionRequest {
common.MsgBase base = 1;
string db_name = 2;
string collection_name = 3;
string partition_name = 4;
int64 dbID = 5;
int64 collectionID = 6;
int64 partitionID = 7;
}
message DropPartitionRequest {
common.MsgBase base = 1;
string db_name = 2;
string collection_name = 3;
string partition_name = 4;
int64 dbID = 5;
int64 collectionID = 6;
int64 partitionID = 7;
}
message TimeTickMsg {
common.MsgBase base = 1;
}
message DataNodeTtMsg {
common.MsgBase base =1;
string channel_name = 2;
uint64 timestamp = 3;
repeated common.SegmentStats segments_stats = 4;
}
././@LongLink 0000644 0000000 0000000 00000000147 00000000000 011605 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/milvus.proto libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/milvus.pro0000664 0001750 0001750 00000145111 15160465216 033022 0 ustar feng feng syntax = "proto3";
package milvus.proto.milvus;
option go_package = "github.com/milvus-io/milvus-proto/go-api/v2/milvuspb";
option java_multiple_files = true;
option java_package = "io.milvus.grpc";
option java_outer_classname = "MilvusProto";
option java_generate_equals_and_hash = true;
option csharp_namespace = "Milvus.Client.Grpc";
import "common.proto";
import "rg.proto";
import "schema.proto";
import "feder.proto";
import "msg.proto";
import "google/protobuf/descriptor.proto";
service MilvusService {
rpc CreateCollection(CreateCollectionRequest) returns (common.Status) {}
rpc DropCollection(DropCollectionRequest) returns (common.Status) {}
rpc HasCollection(HasCollectionRequest) returns (BoolResponse) {}
rpc LoadCollection(LoadCollectionRequest) returns (common.Status) {}
rpc ReleaseCollection(ReleaseCollectionRequest) returns (common.Status) {}
rpc DescribeCollection(DescribeCollectionRequest) returns (DescribeCollectionResponse) {}
rpc GetCollectionStatistics(GetCollectionStatisticsRequest) returns (GetCollectionStatisticsResponse) {}
rpc ShowCollections(ShowCollectionsRequest) returns (ShowCollectionsResponse) {}
rpc AlterCollection(AlterCollectionRequest) returns (common.Status) {}
rpc CreatePartition(CreatePartitionRequest) returns (common.Status) {}
rpc DropPartition(DropPartitionRequest) returns (common.Status) {}
rpc HasPartition(HasPartitionRequest) returns (BoolResponse) {}
rpc LoadPartitions(LoadPartitionsRequest) returns (common.Status) {}
rpc ReleasePartitions(ReleasePartitionsRequest) returns (common.Status) {}
rpc GetPartitionStatistics(GetPartitionStatisticsRequest) returns (GetPartitionStatisticsResponse) {}
rpc ShowPartitions(ShowPartitionsRequest) returns (ShowPartitionsResponse) {}
rpc GetLoadingProgress(GetLoadingProgressRequest) returns (GetLoadingProgressResponse) {}
rpc GetLoadState(GetLoadStateRequest) returns (GetLoadStateResponse) {}
rpc CreateAlias(CreateAliasRequest) returns (common.Status) {}
rpc DropAlias(DropAliasRequest) returns (common.Status) {}
rpc AlterAlias(AlterAliasRequest) returns (common.Status) {}
rpc DescribeAlias(DescribeAliasRequest) returns (DescribeAliasResponse) {}
rpc ListAliases(ListAliasesRequest) returns (ListAliasesResponse) {}
rpc CreateIndex(CreateIndexRequest) returns (common.Status) {}
rpc AlterIndex(AlterIndexRequest) returns (common.Status) {}
rpc DescribeIndex(DescribeIndexRequest) returns (DescribeIndexResponse) {}
rpc GetIndexStatistics(GetIndexStatisticsRequest) returns (GetIndexStatisticsResponse) {}
// Deprecated: use DescribeIndex instead
rpc GetIndexState(GetIndexStateRequest) returns (GetIndexStateResponse) {
option deprecated = true;
}
// Deprecated: use DescribeIndex instead
rpc GetIndexBuildProgress(GetIndexBuildProgressRequest) returns (GetIndexBuildProgressResponse) {
option deprecated = true;
}
rpc DropIndex(DropIndexRequest) returns (common.Status) {}
rpc Insert(InsertRequest) returns (MutationResult) {}
rpc Delete(DeleteRequest) returns (MutationResult) {}
rpc Upsert(UpsertRequest) returns (MutationResult) {}
rpc Search(SearchRequest) returns (SearchResults) {}
rpc HybridSearch(HybridSearchRequest) returns (SearchResults) {}
rpc Flush(FlushRequest) returns (FlushResponse) {}
rpc Query(QueryRequest) returns (QueryResults) {}
rpc CalcDistance(CalcDistanceRequest) returns (CalcDistanceResults) {}
rpc FlushAll(FlushAllRequest) returns (FlushAllResponse) {}
rpc GetFlushState(GetFlushStateRequest) returns (GetFlushStateResponse) {}
rpc GetFlushAllState(GetFlushAllStateRequest) returns (GetFlushAllStateResponse) {}
rpc GetPersistentSegmentInfo(GetPersistentSegmentInfoRequest) returns (GetPersistentSegmentInfoResponse) {}
rpc GetQuerySegmentInfo(GetQuerySegmentInfoRequest) returns (GetQuerySegmentInfoResponse) {}
rpc GetReplicas(GetReplicasRequest) returns (GetReplicasResponse) {}
rpc Dummy(DummyRequest) returns (DummyResponse) {}
// TODO: remove
rpc RegisterLink(RegisterLinkRequest) returns (RegisterLinkResponse) {}
// https://wiki.lfaidata.foundation/display/MIL/MEP+8+--+Add+metrics+for+proxy
rpc GetMetrics(GetMetricsRequest) returns (GetMetricsResponse) {}
rpc GetComponentStates(GetComponentStatesRequest) returns (ComponentStates) {}
rpc LoadBalance(LoadBalanceRequest) returns (common.Status) {}
rpc GetCompactionState(GetCompactionStateRequest) returns (GetCompactionStateResponse) {}
rpc ManualCompaction(ManualCompactionRequest) returns (ManualCompactionResponse) {}
rpc GetCompactionStateWithPlans(GetCompactionPlansRequest) returns (GetCompactionPlansResponse) {}
// https://wiki.lfaidata.foundation/display/MIL/MEP+24+--+Support+bulk+load
rpc Import(ImportRequest) returns (ImportResponse) {}
rpc GetImportState(GetImportStateRequest) returns (GetImportStateResponse) {}
rpc ListImportTasks(ListImportTasksRequest) returns (ListImportTasksResponse) {}
// https://wiki.lfaidata.foundation/display/MIL/MEP+27+--+Support+Basic+Authentication
rpc CreateCredential(CreateCredentialRequest) returns (common.Status) {}
rpc UpdateCredential(UpdateCredentialRequest) returns (common.Status) {}
rpc DeleteCredential(DeleteCredentialRequest) returns (common.Status) {}
rpc ListCredUsers(ListCredUsersRequest) returns (ListCredUsersResponse) {}
// https://wiki.lfaidata.foundation/display/MIL/MEP+29+--+Support+Role-Based+Access+Control
rpc CreateRole(CreateRoleRequest) returns (common.Status) {}
rpc DropRole(DropRoleRequest) returns (common.Status) {}
rpc OperateUserRole(OperateUserRoleRequest) returns (common.Status) {}
rpc SelectRole(SelectRoleRequest) returns (SelectRoleResponse) {}
rpc SelectUser(SelectUserRequest) returns (SelectUserResponse) {}
rpc OperatePrivilege(OperatePrivilegeRequest) returns (common.Status) {}
rpc SelectGrant(SelectGrantRequest) returns (SelectGrantResponse) {}
rpc GetVersion(GetVersionRequest) returns (GetVersionResponse) {}
rpc CheckHealth(CheckHealthRequest) returns (CheckHealthResponse) {}
rpc CreateResourceGroup(CreateResourceGroupRequest) returns (common.Status) {}
rpc DropResourceGroup(DropResourceGroupRequest) returns (common.Status) {}
rpc UpdateResourceGroups(UpdateResourceGroupsRequest) returns (common.Status) {}
rpc TransferNode(TransferNodeRequest) returns (common.Status) {}
rpc TransferReplica(TransferReplicaRequest) returns (common.Status) {}
rpc ListResourceGroups(ListResourceGroupsRequest) returns (ListResourceGroupsResponse) {}
rpc DescribeResourceGroup(DescribeResourceGroupRequest) returns (DescribeResourceGroupResponse) {}
rpc RenameCollection(RenameCollectionRequest) returns (common.Status) {}
rpc ListIndexedSegment(feder.ListIndexedSegmentRequest) returns(feder.ListIndexedSegmentResponse) {}
rpc DescribeSegmentIndexData(feder.DescribeSegmentIndexDataRequest) returns(feder.DescribeSegmentIndexDataResponse) {}
rpc Connect(ConnectRequest) returns (ConnectResponse) {}
rpc AllocTimestamp(AllocTimestampRequest) returns (AllocTimestampResponse) {}
rpc CreateDatabase(CreateDatabaseRequest) returns (common.Status) {}
rpc DropDatabase(DropDatabaseRequest) returns (common.Status) {}
rpc ListDatabases(ListDatabasesRequest) returns (ListDatabasesResponse) {}
rpc AlterDatabase(AlterDatabaseRequest) returns (common.Status) {}
rpc ReplicateMessage(ReplicateMessageRequest) returns (ReplicateMessageResponse) {}
}
message CreateAliasRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeCreateAlias
object_name_index: -1
};
common.MsgBase base = 1;
string db_name = 2;
string collection_name = 3;
string alias = 4;
}
message DropAliasRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeDropAlias
object_name_index: -1
};
common.MsgBase base = 1;
string db_name = 2;
string alias = 3;
}
message AlterAliasRequest{
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeCreateAlias
object_name_index: -1
};
common.MsgBase base = 1;
string db_name = 2;
string collection_name = 3;
string alias = 4;
}
message DescribeAliasRequest{
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeDescribeAlias
object_name_index: -1
};
common.MsgBase base = 1;
string db_name = 2;
string alias = 3;
}
/*
* Describe alias response
*/
message DescribeAliasResponse {
// Response status
common.Status status = 1;
string db_name = 2;
string alias = 3;
string collection = 4;
}
message ListAliasesRequest{
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeListAliases
object_name_index: -1
};
common.MsgBase base = 1;
string db_name = 2;
string collection_name = 3;
}
/*
* List aliases response
*/
message ListAliasesResponse {
// Response status
common.Status status = 1;
string db_name = 2;
string collection_name = 3;
repeated string aliases = 4;
}
/**
* Create collection in milvus
*/
message CreateCollectionRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeCreateCollection
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The unique collection name in milvus.(Required)
string collection_name = 3;
// The serialized `schema.CollectionSchema`(Required)
bytes schema = 4;
// Once set, no modification is allowed (Optional)
// https://github.com/milvus-io/milvus/issues/6690
int32 shards_num = 5;
// The consistency level that the collection used, modification is not supported now.
common.ConsistencyLevel consistency_level = 6;
repeated common.KeyValuePair properties = 7;
int64 num_partitions = 8; // num of default physical partitions, only used in partition key mode and changes are not supported
}
/**
* Drop collection in milvus, also will drop data in collection.
*/
message DropCollectionRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeDropCollection
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The unique collection name in milvus.(Required)
string collection_name = 3;
}
/**
* Alter collection in milvus
*/
message AlterCollectionRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeCreateCollection
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The unique collection name in milvus.(Required)
string collection_name = 3;
int64 collectionID = 4;
repeated common.KeyValuePair properties = 5;
}
/**
* Check collection exist in milvus or not.
*/
message HasCollectionRequest {
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name you want to check.
string collection_name = 3;
// If time_stamp is not zero, will return true when time_stamp >= created collection timestamp, otherwise will return false.
uint64 time_stamp = 4;
}
message BoolResponse {
common.Status status = 1;
bool value = 2;
}
message StringResponse {
common.Status status = 1;
string value = 2;
}
/**
* Get collection meta datas like: schema, collectionID, shards number ...
*/
message DescribeCollectionRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeDescribeCollection
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name you want to describe, you can pass collection_name or collectionID
string collection_name = 3;
// The collection ID you want to describe
int64 collectionID = 4;
// If time_stamp is not zero, will describe collection success when time_stamp >= created collection timestamp, otherwise will throw error.
uint64 time_stamp = 5;
}
/**
* DescribeCollection Response
*/
message DescribeCollectionResponse {
// Contain error_code and reason
common.Status status = 1;
// The schema param when you created collection.
schema.CollectionSchema schema = 2;
// The collection id
int64 collectionID = 3;
// System design related, users should not perceive
repeated string virtual_channel_names = 4;
// System design related, users should not perceive
repeated string physical_channel_names = 5;
// Hybrid timestamp in milvus
uint64 created_timestamp = 6;
// The utc timestamp calculated by created_timestamp
uint64 created_utc_timestamp = 7;
// The shards number you set.
int32 shards_num = 8;
// The aliases of this collection
repeated string aliases = 9;
// The message ID/posititon when collection is created
repeated common.KeyDataPair start_positions = 10;
// The consistency level that the collection used, modification is not supported now.
common.ConsistencyLevel consistency_level = 11;
// The collection name
string collection_name = 12;
repeated common.KeyValuePair properties = 13;
string db_name = 14;
int64 num_partitions = 15;
int64 db_id = 16;
}
/**
* Load collection data into query nodes, then you can do vector search on this collection.
*/
message LoadCollectionRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeLoad
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name you want to load
string collection_name = 3;
// The replica number to load, default by 1
int32 replica_number = 4;
// create replica used resource group
repeated string resource_groups = 5;
// Whether to enable refresh mode.
bool refresh = 6;
}
/**
* Release collection data from query nodes, then you can't do vector search on this collection.
*/
message ReleaseCollectionRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeRelease
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name you want to release
string collection_name = 3;
}
/**
* Get statistics like row_count.
* WARNING: This API is experimental and not useful for now.
*/
message GetStatisticsRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeGetStatistics
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name you want get statistics
string collection_name = 3;
// The partition names you want get statistics, empty for all partitions
repeated string partition_names = 4;
// Not useful for now, reserved for future
uint64 guarantee_timestamp = 5;
}
/**
* Will return statistics in stats field like [{key:"row_count",value:"1"}]
* WARNING: This API is experimental and not useful for now.
*/
message GetStatisticsResponse {
// Contain error_code and reason
common.Status status = 1;
// Collection statistics data
repeated common.KeyValuePair stats = 2;
}
/**
* Get collection statistics like row_count.
*/
message GetCollectionStatisticsRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeGetStatistics
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name you want get statistics
string collection_name = 3;
}
/**
* Will return collection statistics in stats field like [{key:"row_count",value:"1"}]
*/
message GetCollectionStatisticsResponse {
// Contain error_code and reason
common.Status status = 1;
// Collection statistics data
repeated common.KeyValuePair stats = 2;
}
/*
* This is for ShowCollectionsRequest type field.
*/
// Deprecated: use GetLoadingProgress rpc instead
enum ShowType {
option deprecated = true;
// Will return all collections
All = 0;
// Will return loaded collections with their inMemory_percentages
InMemory = 1;
}
/*
* List collections
*/
message ShowCollectionsRequest {
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// Not useful for now
uint64 time_stamp = 3;
// Decide return Loaded collections or All collections(Optional)
ShowType type = 4;
// When type is InMemory, will return these collection's inMemory_percentages.(Optional)
// Deprecated: use GetLoadingProgress rpc instead
repeated string collection_names = 5 [deprecated=true];
}
/*
* Return basic collection infos.
*/
message ShowCollectionsResponse {
// Contain error_code and reason
common.Status status = 1;
// Collection name array
repeated string collection_names = 2;
// Collection Id array
repeated int64 collection_ids = 3;
// Hybrid timestamps in milvus
repeated uint64 created_timestamps = 4;
// The utc timestamp calculated by created_timestamp
repeated uint64 created_utc_timestamps = 5;
// Load percentage on querynode when type is InMemory
// Deprecated: use GetLoadingProgress rpc instead
repeated int64 inMemory_percentages = 6 [deprecated=true];
// Indicate whether query service is available
repeated bool query_service_available = 7;
}
/*
* Create partition in created collection.
*/
message CreatePartitionRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeCreatePartition
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name in milvus
string collection_name = 3;
// The partition name you want to create.
string partition_name = 4;
}
/*
* Drop partition in created collection.
*/
message DropPartitionRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeDropPartition
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name in milvus
string collection_name = 3;
// The partition name you want to drop
string partition_name = 4;
}
/*
* Check if partition exist in collection or not.
*/
message HasPartitionRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeHasPartition
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name in milvus
string collection_name = 3;
// The partition name you want to check
string partition_name = 4;
}
/*
* Load specific partitions data of one collection into query nodes
* Then you can get these data as result when you do vector search on this collection.
*/
message LoadPartitionsRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeLoad
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name in milvus
string collection_name = 3;
// The partition names you want to load
repeated string partition_names = 4;
// The replicas number you would load, 1 by default
int32 replica_number = 5;
// create replica used resource group
repeated string resource_groups = 6;
// Whether to enable refresh mode.
bool refresh = 7;
}
/*
* Release specific partitions data of one collection from query nodes.
* Then you can not get these data as result when you do vector search on this collection.
*/
message ReleasePartitionsRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeRelease
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name in milvus
string collection_name = 3;
// The partition names you want to release
repeated string partition_names = 4;
}
/*
* Get partition statistics like row_count.
*/
message GetPartitionStatisticsRequest {
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name in milvus
string collection_name = 3;
// The partition name you want to collect statistics
string partition_name = 4;
}
message GetPartitionStatisticsResponse {
common.Status status = 1;
repeated common.KeyValuePair stats = 2;
}
/*
* List all partitions for particular collection
*/
message ShowPartitionsRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeShowPartitions
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The collection name you want to describe, you can pass collection_name or collectionID
string collection_name = 3;
// The collection id in milvus
int64 collectionID = 4;
// When type is InMemory, will return these patitions's inMemory_percentages.(Optional)
repeated string partition_names = 5;
// Decide return Loaded partitions or All partitions(Optional)
// Deprecated: use GetLoadingProgress rpc instead
ShowType type = 6 [deprecated=true];
}
/*
* List all partitions for particular collection response.
* The returned datas are all rows, we can format to columns by therir index.
*/
message ShowPartitionsResponse {
// Contain error_code and reason
common.Status status = 1;
// All partition names for this collection
repeated string partition_names = 2;
// All partition ids for this collection
repeated int64 partitionIDs = 3;
// All hybrid timestamps
repeated uint64 created_timestamps = 4;
// All utc timestamps calculated by created_timestamps
repeated uint64 created_utc_timestamps = 5;
// Load percentage on querynode
// Deprecated: use GetLoadingProgress rpc instead
repeated int64 inMemory_percentages = 6 [deprecated=true];
}
message DescribeSegmentRequest {
common.MsgBase base = 1;
int64 collectionID = 2;
int64 segmentID = 3;
}
message DescribeSegmentResponse {
common.Status status = 1;
int64 indexID = 2;
int64 buildID = 3;
bool enable_index = 4;
int64 fieldID = 5;
}
message ShowSegmentsRequest {
common.MsgBase base = 1;
int64 collectionID = 2;
int64 partitionID = 3;
}
message ShowSegmentsResponse {
common.Status status = 1;
repeated int64 segmentIDs = 2;
}
/*
* Create index for vector datas
*/
message CreateIndexRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeCreateIndex
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The particular collection name you want to create index.
string collection_name = 3;
// The vector field name in this particular collection
string field_name = 4;
// Support keys: index_type,metric_type, params. Different index_type may has different params.
repeated common.KeyValuePair extra_params = 5;
// Version before 2.0.2 doesn't contain index_name, we use default index name.
string index_name = 6;
}
/*
* Alter index
*/
message AlterIndexRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeCreateIndex
object_name_index: 3
};
common.MsgBase base = 1;
string db_name = 2;
string collection_name = 3;
string index_name = 4;
repeated common.KeyValuePair extra_params = 5;
}
/*
* Get created index information.
* Current release of Milvus only supports showing latest built index.
*/
message DescribeIndexRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeIndexDetail
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2;
// The particular collection name in Milvus
string collection_name = 3;
// The vector field name in this particular collection
string field_name = 4;
// No need to set up for now @2021.06.30
string index_name = 5;
uint64 timestamp = 6;
}
/*
* Index informations
*/
message IndexDescription {
// Index name
string index_name = 1;
// Index id
int64 indexID = 2;
// Will return index_type, metric_type, params(like nlist).
repeated common.KeyValuePair params = 3;
// The vector field name
string field_name = 4;
// index build progress
int64 indexed_rows = 5;
int64 total_rows = 6;
// index state
common.IndexState state = 7;
string index_state_fail_reason = 8;
int64 pending_index_rows = 9;
}
/*
* Describe index response
*/
message DescribeIndexResponse {
// Response status
common.Status status = 1;
// All index informations, for now only return tha latest index you created for the collection.
repeated IndexDescription index_descriptions = 2;
}
/*
* Get index building progress
*/
message GetIndexBuildProgressRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeIndexDetail
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
string db_name = 2 ;
// The collection name in milvus
string collection_name = 3;
// The vector field name in this collection
string field_name = 4;
// Not useful for now
string index_name = 5;
}
message GetIndexBuildProgressResponse {
common.Status status = 1;
int64 indexed_rows = 2;
int64 total_rows = 3;
}
message GetIndexStateRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeIndexDetail
object_name_index: 3
};
common.MsgBase base = 1; // must
string db_name = 2 ;
string collection_name = 3; // must
string field_name = 4;
string index_name = 5; // No need to set up for now @2021.06.30
}
message GetIndexStateResponse {
common.Status status = 1;
common.IndexState state = 2;
string fail_reason = 3;
}
message DropIndexRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeDropIndex
object_name_index: 3
};
common.MsgBase base = 1; // must
string db_name = 2;
string collection_name = 3; // must
// Deprecated: not be used in the milvus
string field_name = 4;
string index_name = 5;
}
message InsertRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeInsert
object_name_index: 3
};
common.MsgBase base = 1;
string db_name = 2;
string collection_name = 3;
string partition_name = 4;
repeated schema.FieldData fields_data = 5;
repeated uint32 hash_keys = 6;
uint32 num_rows = 7;
}
message UpsertRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeUpsert
object_name_index: 3
};
common.MsgBase base = 1;
string db_name = 2;
string collection_name = 3;
string partition_name = 4;
repeated schema.FieldData fields_data = 5;
repeated uint32 hash_keys = 6;
uint32 num_rows = 7;
}
message MutationResult {
common.Status status = 1;
schema.IDs IDs = 2; // required for insert, delete, upsert
repeated uint32 succ_index = 3; // error indexes indicate
repeated uint32 err_index = 4; // error indexes indicate
bool acknowledged = 5;
int64 insert_cnt = 6;
int64 delete_cnt = 7;
int64 upsert_cnt = 8;
uint64 timestamp = 9;
}
message DeleteRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeDelete
object_name_index: 3
};
common.MsgBase base = 1;
string db_name = 2;
string collection_name = 3;
string partition_name = 4;
string expr = 5;
repeated uint32 hash_keys = 6;
common.ConsistencyLevel consistency_level = 7;
}
message SubSearchRequest {
string dsl = 1; // must
// serialized `PlaceholderGroup`
bytes placeholder_group = 2; // must
common.DslType dsl_type = 3; // must
repeated common.KeyValuePair search_params = 4; // must
int64 nq = 5;
}
message SearchRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeSearch
object_name_index: 3
};
common.MsgBase base = 1; // must
string db_name = 2;
string collection_name = 3; // must
repeated string partition_names = 4; // must
string dsl = 5; // must
// serialized `PlaceholderGroup`
bytes placeholder_group = 6; // must
common.DslType dsl_type = 7; // must
repeated string output_fields = 8;
repeated common.KeyValuePair search_params = 9; // must
uint64 travel_timestamp = 10;
uint64 guarantee_timestamp = 11; // guarantee_timestamp
int64 nq = 12;
bool not_return_all_meta = 13;
common.ConsistencyLevel consistency_level = 14;
bool use_default_consistency = 15;
bool search_by_primary_keys = 16;
repeated SubSearchRequest sub_reqs = 17;
}
message Hits {
repeated int64 IDs = 1;
repeated bytes row_data = 2;
repeated float scores = 3;
}
message SearchResults {
common.Status status = 1;
schema.SearchResultData results = 2;
string collection_name = 3;
}
message HybridSearchRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeSearch
object_name_index: 3
};
common.MsgBase base = 1; // must
string db_name = 2;
string collection_name = 3; // must
repeated string partition_names = 4; // must
repeated SearchRequest requests = 5;
repeated common.KeyValuePair rank_params = 6; // must
uint64 travel_timestamp = 7;
uint64 guarantee_timestamp = 8; // guarantee_timestamp
bool not_return_all_meta = 9;
repeated string output_fields = 10;
common.ConsistencyLevel consistency_level = 11;
bool use_default_consistency = 12;
}
message FlushRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeFlush
object_name_indexs: 3
};
common.MsgBase base = 1;
string db_name = 2;
repeated string collection_names = 3;
}
message FlushResponse{
common.Status status = 1;
string db_name = 2;
map coll_segIDs = 3;
map flush_coll_segIDs = 4;
map coll_seal_times = 5; // physical time for backup tool
map coll_flush_ts = 6; // hybrid ts for geting flush tate
map channel_cps = 7;
}
message QueryRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeQuery
object_name_index: 3
};
common.MsgBase base = 1;
string db_name = 2;
string collection_name = 3;
string expr = 4;
repeated string output_fields = 5;
repeated string partition_names = 6;
uint64 travel_timestamp = 7;
uint64 guarantee_timestamp = 8; // guarantee_timestamp
repeated common.KeyValuePair query_params = 9; // optional
bool not_return_all_meta = 10;
common.ConsistencyLevel consistency_level = 11;
bool use_default_consistency = 12;
}
message QueryResults {
common.Status status = 1;
repeated schema.FieldData fields_data = 2;
string collection_name = 3;
repeated string output_fields = 4;
}
message VectorIDs {
string collection_name = 1;
string field_name = 2;
schema.IDs id_array = 3;
repeated string partition_names = 4;
}
message VectorsArray {
oneof array {
VectorIDs id_array = 1; // vector ids
schema.VectorField data_array = 2; // vectors data
}
}
message CalcDistanceRequest {
common.MsgBase base = 1;
VectorsArray op_left = 2; // vectors on the left of operator
VectorsArray op_right = 3; // vectors on the right of operator
repeated common.KeyValuePair params = 4; // "metric":"L2"/"IP"/"HAMMIN"/"TANIMOTO"
}
message CalcDistanceResults {
common.Status status = 1;
// num(op_left)*num(op_right) distance values, "HAMMIN" return integer distance
oneof array {
schema.IntArray int_dist = 2;
schema.FloatArray float_dist = 3;
}
}
message FlushAllRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeFlushAll
object_name_index: -1
};
common.MsgBase base = 1;
string db_name = 2;
}
message FlushAllResponse {
common.Status status = 1;
uint64 flush_all_ts = 2;
}
message PersistentSegmentInfo {
int64 segmentID = 1;
int64 collectionID = 2;
int64 partitionID = 3;
int64 num_rows = 4;
common.SegmentState state = 5;
}
message GetPersistentSegmentInfoRequest {
common.MsgBase base = 1; // must
string dbName = 2;
string collectionName = 3; // must
}
message GetPersistentSegmentInfoResponse {
common.Status status = 1;
repeated PersistentSegmentInfo infos = 2;
}
message QuerySegmentInfo {
int64 segmentID = 1;
int64 collectionID = 2;
int64 partitionID = 3;
int64 mem_size = 4;
int64 num_rows = 5;
string index_name = 6;
int64 indexID = 7;
// deprecated, check node_ids(NodeIds) field
int64 nodeID = 8 [deprecated=true];
common.SegmentState state = 9;
repeated int64 nodeIds = 10;
}
message GetQuerySegmentInfoRequest {
common.MsgBase base = 1; // must
string dbName = 2;
string collectionName = 3; // must
}
message GetQuerySegmentInfoResponse {
common.Status status = 1;
repeated QuerySegmentInfo infos = 2;
}
message DummyRequest {
string request_type = 1;
}
message DummyResponse {
string response = 1;
}
message RegisterLinkRequest {
}
message RegisterLinkResponse {
common.Address address = 1;
common.Status status = 2;
}
message GetMetricsRequest {
common.MsgBase base = 1;
string request = 2; // request is of jsonic format
}
message GetMetricsResponse {
common.Status status = 1;
string response = 2; // response is of jsonic format
string component_name = 3; // metrics from which component
}
message ComponentInfo {
int64 nodeID = 1;
string role = 2;
common.StateCode state_code = 3;
repeated common.KeyValuePair extra_info = 4;
}
message ComponentStates {
ComponentInfo state = 1;
repeated ComponentInfo subcomponent_states = 2;
common.Status status = 3;
}
message GetComponentStatesRequest {
}
/*
* Do load balancing operation from src_nodeID to dst_nodeID.
*/
message LoadBalanceRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeLoadBalance
object_name_index: 5
};
common.MsgBase base = 1;
int64 src_nodeID = 2;
repeated int64 dst_nodeIDs = 3;
repeated int64 sealed_segmentIDs = 4;
string collectionName = 5;
string db_name = 6;
}
message ManualCompactionRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeCompaction
object_name_index: 1
};
int64 collectionID = 1;
uint64 timetravel = 2;
bool majorCompaction = 3;
}
message ManualCompactionResponse {
common.Status status = 1;
int64 compactionID = 2;
int32 compactionPlanCount = 3;
}
message GetCompactionStateRequest {
int64 compactionID = 1;
}
message GetCompactionStateResponse {
common.Status status = 1;
common.CompactionState state = 2;
int64 executingPlanNo = 3;
int64 timeoutPlanNo = 4;
int64 completedPlanNo = 5;
int64 failedPlanNo = 6;
}
message GetCompactionPlansRequest {
int64 compactionID = 1;
}
message GetCompactionPlansResponse {
common.Status status = 1;
common.CompactionState state = 2;
repeated CompactionMergeInfo mergeInfos = 3;
}
message CompactionMergeInfo {
repeated int64 sources = 1;
int64 target = 2;
}
message GetFlushStateRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeGetFlushState
object_name_index: 4
};
repeated int64 segmentIDs = 1;
uint64 flush_ts = 2;
string db_name = 3;
string collection_name = 4;
}
message GetFlushStateResponse {
common.Status status = 1;
bool flushed = 2;
}
message GetFlushAllStateRequest {
common.MsgBase base = 1;
uint64 flush_all_ts = 2;
string db_name = 3;
}
message GetFlushAllStateResponse {
common.Status status = 1;
bool flushed = 2;
}
message ImportRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeImport
object_name_index: 1
};
string collection_name = 1; // target collection
string partition_name = 2; // target partition
repeated string channel_names = 3; // channel names for the collection
bool row_based = 4; // the file is row-based or column-based
repeated string files = 5; // file paths to be imported
repeated common.KeyValuePair options = 6; // import options, bucket, etc.
string db_name = 7; // target database
bytes clustering_info = 8; // serialized `schema.ClusteringInfo`
}
message ImportResponse {
common.Status status = 1;
repeated int64 tasks = 2; // id array of import tasks
}
message GetImportStateRequest {
int64 task = 1; // id of an import task
}
message GetImportStateResponse {
common.Status status = 1;
common.ImportState state = 2; // is this import task finished or not
int64 row_count = 3; // if the task is finished, this value is how many rows are imported. if the task is not finished, this value is how many rows are parsed. return 0 if failed.
repeated int64 id_list = 4; // auto generated ids if the primary key is autoid
repeated common.KeyValuePair infos = 5; // more information about the task, progress percent, file path, failed reason, etc.
int64 id = 6; // id of an import task
int64 collection_id = 7; // collection ID of the import task.
repeated int64 segment_ids = 8; // a list of segment IDs created by the import task.
int64 create_ts = 9; // timestamp when the import task is created.
}
message ListImportTasksRequest {
string collection_name = 1; // target collection, list all tasks if the name is empty
int64 limit = 2; // maximum number of tasks returned, list all tasks if the value is 0
string db_name = 3;
}
message ListImportTasksResponse {
common.Status status = 1;
repeated GetImportStateResponse tasks = 2; // list of all import tasks
}
message GetReplicasRequest {
common.MsgBase base = 1;
int64 collectionID = 2;
bool with_shard_nodes = 3;
string collection_name = 4;
string db_name = 5;
}
message GetReplicasResponse {
common.Status status = 1;
repeated ReplicaInfo replicas = 2;
}
message ReplicaInfo { // ReplicaGroup
int64 replicaID = 1;
int64 collectionID = 2;
repeated int64 partition_ids = 3; // empty indicates to load collection
repeated ShardReplica shard_replicas = 4;
repeated int64 node_ids = 5; // include leaders
string resource_group_name = 6;
// outbound access rg -> node num
map num_outbound_node = 7;
}
message ShardReplica {
int64 leaderID = 1;
string leader_addr = 2; // IP:port
string dm_channel_name = 3;
// optional, DO NOT save it in meta, set it only for GetReplicas()
// if with_shard_nodes is true
repeated int64 node_ids = 4;
}
service ProxyService {
rpc RegisterLink(RegisterLinkRequest) returns (RegisterLinkResponse) {}
}
// https://wiki.lfaidata.foundation/display/MIL/MEP+27+--+Support+Basic+Authentication
message CreateCredentialRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeCreateOwnership
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
// username
string username = 2;
// ciphertext password
string password = 3;
// create time
uint64 created_utc_timestamps = 4;
// modify time
uint64 modified_utc_timestamps = 5;
}
message UpdateCredentialRequest {
option (common.privilege_ext_obj) = {
object_type: User
object_privilege: PrivilegeUpdateUser
object_name_index: 2
};
// Not useful for now
common.MsgBase base = 1;
// username
string username = 2;
// old password
string oldPassword = 3;
// new password
string newPassword = 4;
// create time
uint64 created_utc_timestamps = 5;
// modify time
uint64 modified_utc_timestamps = 6;
}
message DeleteCredentialRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeDropOwnership
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
// Not useful for now
string username = 2;
}
message ListCredUsersResponse {
// Contain error_code and reason
common.Status status = 1;
// username array
repeated string usernames = 2;
}
message ListCredUsersRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeSelectOwnership
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
}
// https://wiki.lfaidata.foundation/display/MIL/MEP+29+--+Support+Role-Based+Access+Control
message RoleEntity {
string name = 1;
}
message UserEntity {
string name = 1;
}
message CreateRoleRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeCreateOwnership
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
// role
RoleEntity entity = 2;
}
message DropRoleRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeDropOwnership
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
// role name
string role_name = 2;
}
enum OperateUserRoleType {
AddUserToRole = 0;
RemoveUserFromRole = 1;
}
message OperateUserRoleRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeManageOwnership
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
// username
string username = 2;
// role name
string role_name = 3;
// operation type
OperateUserRoleType type = 4;
}
message SelectRoleRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeSelectOwnership
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
// role
RoleEntity role = 2;
// include user info
bool include_user_info = 3;
}
message RoleResult {
RoleEntity role = 1;
repeated UserEntity users = 2;
}
message SelectRoleResponse {
// Not useful for now
common.Status status = 1;
// role result array
repeated RoleResult results = 2;
}
message SelectUserRequest {
option (common.privilege_ext_obj) = {
object_type: User
object_privilege: PrivilegeSelectUser
object_name_index: 2
};
// Not useful for now
common.MsgBase base = 1;
// user
UserEntity user = 2;
// include user info
bool include_role_info = 3;
}
message UserResult {
UserEntity user = 1;
repeated RoleEntity roles = 2;
}
message SelectUserResponse {
// Not useful for now
common.Status status = 1;
// user result array
repeated UserResult results = 2;
}
message ObjectEntity {
string name = 1;
}
message PrivilegeEntity {
string name = 1;
}
message GrantorEntity {
UserEntity user = 1;
PrivilegeEntity privilege = 2;
}
message GrantPrivilegeEntity {
repeated GrantorEntity entities = 1;
}
message GrantEntity {
// role
RoleEntity role = 1;
// object
ObjectEntity object = 2;
// object name
string object_name = 3;
// privilege
GrantorEntity grantor = 4;
// db name
string db_name = 5;
}
message SelectGrantRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeSelectOwnership
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
// grant
GrantEntity entity = 2;
}
message SelectGrantResponse {
// Not useful for now
common.Status status = 1;
// grant info array
repeated GrantEntity entities = 2;
}
enum OperatePrivilegeType {
Grant = 0;
Revoke = 1;
}
message OperatePrivilegeRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeManageOwnership
object_name_index: -1
};
// Not useful for now
common.MsgBase base = 1;
// grant
GrantEntity entity = 2;
// operation type
OperatePrivilegeType type = 3;
}
message GetLoadingProgressRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeLoad
object_name_index: 2
};
// Not useful for now
common.MsgBase base = 1;
string collection_name = 2;
repeated string partition_names = 3;
string db_name = 4;
}
message GetLoadingProgressResponse {
common.Status status = 1;
int64 progress = 2;
int64 refresh_progress = 3;
}
message GetLoadStateRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeLoad
object_name_index: 2
};
// Not useful for now
common.MsgBase base = 1;
string collection_name = 2;
repeated string partition_names = 3;
string db_name = 4;
}
message GetLoadStateResponse {
common.Status status = 1;
common.LoadState state = 2;
}
message MilvusExt {
string version = 1;
}
extend google.protobuf.FileOptions {
MilvusExt milvus_ext_obj = 1001;
}
message GetVersionRequest {
}
message GetVersionResponse {
common.Status status = 1;
string version =2;
}
enum QuotaState {
Unknown = 0;
ReadLimited = 2;
WriteLimited = 3;
DenyToRead = 4;
DenyToWrite = 5;
}
message CheckHealthRequest {
}
message CheckHealthResponse {
common.Status status = 1;
bool isHealthy = 2;
repeated string reasons = 3;
repeated QuotaState quota_states = 4;
}
message CreateResourceGroupRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeCreateResourceGroup
object_name_index: -1
};
common.MsgBase base = 1;
string resource_group = 2;
rg.ResourceGroupConfig config = 3;
}
message UpdateResourceGroupsRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeUpdateResourceGroups
object_name_index: -1
};
common.MsgBase base = 1;
map resource_groups = 2;
}
message DropResourceGroupRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeDropResourceGroup
object_name_index: -1
};
common.MsgBase base = 1;
string resource_group = 2;
}
// transfer `nodeNum` nodes from `source_resource_group` to `target_resource_group`
message TransferNodeRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeTransferNode
object_name_index: -1
};
common.MsgBase base = 1;
string source_resource_group = 2;
string target_resource_group = 3;
int32 num_node = 4;
}
// transfer `replicaNum` replicas in `collectionID` from `source_resource_group` to `target_resource_group`
message TransferReplicaRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeTransferReplica
object_name_index: -1
};
common.MsgBase base = 1;
string source_resource_group = 2;
string target_resource_group = 3;
string collection_name = 4;
int64 num_replica = 5;
string db_name = 6;
}
message ListResourceGroupsRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeListResourceGroups
object_name_index: -1
};
common.MsgBase base = 1;
}
message ListResourceGroupsResponse {
common.Status status = 1;
repeated string resource_groups = 2;
}
message DescribeResourceGroupRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeDescribeResourceGroup
object_name_index: -1
};
common.MsgBase base = 1;
string resource_group = 2;
}
message DescribeResourceGroupResponse {
common.Status status = 1;
ResourceGroup resource_group = 2;
}
message ResourceGroup {
string name = 1;
int32 capacity = 2;
int32 num_available_node = 3;
// collection name -> loaded replica num
map num_loaded_replica = 4;
// collection name -> accessed other rg's node num
map num_outgoing_node = 5;
// collection name -> be accessed node num by other rg
map num_incoming_node = 6;
// resource group configuration.
rg.ResourceGroupConfig config = 7;
// query node belong to this resource group now.
repeated common.NodeInfo nodes = 8;
}
message RenameCollectionRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeRenameCollection
object_name_index: -1
};
common.MsgBase base = 1;
string db_name = 2;
string oldName = 3;
string newName = 4;
string newDBName =5;
}
message GetIndexStatisticsRequest {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeIndexDetail
object_name_index: 3
};
// Not useful for now
common.MsgBase base = 1;
// Not useful for now
string db_name = 2;
// The particular collection name in Milvus
string collection_name = 3;
// The index name in this particular collection
string index_name = 4;
uint64 timestamp = 5;
}
message GetIndexStatisticsResponse {
// Response status
common.Status status = 1;
// All index information.
repeated IndexDescription index_descriptions = 2;
}
message ConnectRequest {
common.MsgBase base = 1;
common.ClientInfo client_info = 2;
}
message ConnectResponse {
common.Status status = 1;
common.ServerInfo server_info = 2;
int64 identifier = 3;
}
message AllocTimestampRequest {
common.MsgBase base = 1;
}
message AllocTimestampResponse {
common.Status status = 1;
uint64 timestamp = 2;
}
message CreateDatabaseRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeCreateDatabase
object_name_index: -1
};
common.MsgBase base = 1;
string db_name = 2;
}
message DropDatabaseRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeDropDatabase
object_name_index: -1
};
common.MsgBase base = 1;
string db_name = 2;
}
message ListDatabasesRequest {
common.MsgBase base = 1;
}
message ListDatabasesResponse {
common.Status status = 1;
repeated string db_names = 2;
repeated uint64 created_timestamp = 3;
}
message AlterDatabaseRequest {
option (common.privilege_ext_obj) = {
object_type: Global
object_privilege: PrivilegeAlterDatabase
object_name_index: -1
};
common.MsgBase base = 1;
string db_name = 2;
string db_id = 3;
repeated common.KeyValuePair properties = 4;
}
message ReplicateMessageRequest {
common.MsgBase base = 1;
string channel_name = 2;
uint64 BeginTs = 3;
uint64 EndTs = 4;
repeated bytes Msgs = 5;
repeated msg.MsgPosition StartPositions = 6;
repeated msg.MsgPosition EndPositions = 7;
}
message ReplicateMessageResponse {
common.Status status = 1;
string position = 2;
}
message ImportAuthPlaceholder {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeImport
object_name_index: 1
};
string db_name = 1;
string collection_name = 2;
string partition_name = 3;
}
message GetImportProgressAuthPlaceholder {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeImport
object_name_index: 1
};
string db_name = 1;
}
message ListImportsAuthPlaceholder {
option (common.privilege_ext_obj) = {
object_type: Collection
object_privilege: PrivilegeImport
object_name_index: 1
};
string db_name = 3;
string collection_name = 1;
}
././@LongLink 0000644 0000000 0000000 00000000150 00000000000 011577 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/segcore.proto libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/segcore.pr0000664 0001750 0001750 00000001760 15160465216 032754 0 ustar feng feng syntax = "proto3";
package milvus.proto.segcore;
option go_package = "github.com/milvus-io/milvus/internal/proto/segcorepb";
import "schema.proto";
import "common.proto";
message RetrieveResults {
schema.IDs ids = 1;
repeated int64 offset = 2;
repeated schema.FieldData fields_data = 3;
int64 all_retrieve_count = 4;
}
message LoadFieldMeta {
int64 min_timestamp = 1;
int64 max_timestamp = 2;
int64 row_count = 3;
}
message LoadSegmentMeta {
// TODOs
repeated LoadFieldMeta metas = 1;
int64 total_size = 2;
}
message InsertRecord {
repeated schema.FieldData fields_data = 1;
int64 num_rows = 2;
}
message FieldIndexMeta {
int64 fieldID = 1;
int64 collectionID = 2;
string index_name = 3;
repeated common.KeyValuePair type_params = 4;
repeated common.KeyValuePair index_params = 5;
bool is_auto_index = 6;
repeated common.KeyValuePair user_index_params = 7;
}
message CollectionIndexMeta {
int64 maxIndexRowCount = 1;
repeated FieldIndexMeta index_metas = 2;
} libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/plan.proto0000664 0001750 0001750 00000010114 15160465216 032772 0 ustar feng feng syntax = "proto3";
package milvus.proto.plan;
option go_package = "github.com/milvus-io/milvus/internal/proto/planpb";
import "schema.proto";
enum OpType {
Invalid = 0;
GreaterThan = 1;
GreaterEqual = 2;
LessThan = 3;
LessEqual = 4;
Equal = 5;
NotEqual = 6;
PrefixMatch = 7; // startsWith
PostfixMatch = 8; // endsWith
Match = 9; // like
Range = 10; // for case 1 < a < b
In = 11; // TODO:: used for term expr
NotIn = 12;
};
enum ArithOpType {
Unknown = 0;
Add = 1;
Sub = 2;
Mul = 3;
Div = 4;
Mod = 5;
ArrayLength = 6;
};
enum VectorType {
BinaryVector = 0;
FloatVector = 1;
Float16Vector = 2;
BFloat16Vector = 3;
SparseFloatVector = 4;
};
message GenericValue {
oneof val {
bool bool_val = 1;
int64 int64_val = 2;
double float_val = 3;
string string_val = 4;
Array array_val = 5;
};
}
message Array {
repeated GenericValue array = 1;
bool same_type = 2;
schema.DataType element_type = 3;
}
message QueryInfo {
int64 topk = 1;
string metric_type = 3;
string search_params = 4;
int64 round_decimal = 5;
int64 group_by_field_id = 6;
bool materialized_view_involved = 7;
}
message ColumnInfo {
int64 field_id = 1;
schema.DataType data_type = 2;
bool is_primary_key = 3;
bool is_autoID = 4;
repeated string nested_path = 5;
bool is_partition_key = 6;
schema.DataType element_type = 7;
bool is_clustering_key = 8;
}
message ColumnExpr {
ColumnInfo info = 1;
}
message ExistsExpr {
ColumnInfo info = 1;
}
message ValueExpr {
GenericValue value = 1;
}
message UnaryRangeExpr {
ColumnInfo column_info = 1;
OpType op = 2;
GenericValue value = 3;
}
message BinaryRangeExpr {
ColumnInfo column_info = 1;
bool lower_inclusive = 2;
bool upper_inclusive = 3;
GenericValue lower_value = 4;
GenericValue upper_value = 5;
}
message CompareExpr {
ColumnInfo left_column_info = 1;
ColumnInfo right_column_info = 2;
OpType op = 3;
}
message TermExpr {
ColumnInfo column_info = 1;
repeated GenericValue values = 2;
bool is_in_field = 3;
}
message JSONContainsExpr {
ColumnInfo column_info = 1;
repeated GenericValue elements = 2;
// 0: invalid
// 1: json_contains | array_contains
// 2: json_contains_all | array_contains_all
// 3: json_contains_any | array_contains_any
enum JSONOp {
Invalid = 0;
Contains = 1;
ContainsAll = 2;
ContainsAny = 3;
}
JSONOp op = 3;
bool elements_same_type = 4;
}
message UnaryExpr {
enum UnaryOp {
Invalid = 0;
Not = 1;
};
UnaryOp op = 1;
Expr child = 2;
}
message BinaryExpr {
enum BinaryOp {
Invalid = 0;
LogicalAnd = 1;
LogicalOr = 2;
}
BinaryOp op = 1;
Expr left = 2;
Expr right = 3;
}
message BinaryArithOp {
ColumnInfo column_info = 1;
ArithOpType arith_op = 2;
GenericValue right_operand = 3;
}
message BinaryArithExpr {
Expr left = 1;
Expr right = 2;
ArithOpType op = 3;
}
message BinaryArithOpEvalRangeExpr {
ColumnInfo column_info = 1;
ArithOpType arith_op = 2;
GenericValue right_operand = 3;
OpType op = 4;
GenericValue value = 5;
}
message AlwaysTrueExpr {}
message Expr {
oneof expr {
TermExpr term_expr = 1;
UnaryExpr unary_expr = 2;
BinaryExpr binary_expr = 3;
CompareExpr compare_expr = 4;
UnaryRangeExpr unary_range_expr = 5;
BinaryRangeExpr binary_range_expr = 6;
BinaryArithOpEvalRangeExpr binary_arith_op_eval_range_expr = 7;
BinaryArithExpr binary_arith_expr = 8;
ValueExpr value_expr = 9;
ColumnExpr column_expr = 10;
ExistsExpr exists_expr = 11;
AlwaysTrueExpr always_true_expr = 12;
JSONContainsExpr json_contains_expr = 13;
};
}
message VectorANNS {
VectorType vector_type = 1;
int64 field_id = 2;
Expr predicates = 3;
QueryInfo query_info = 4;
string placeholder_tag = 5; // always be "$0"
}
message QueryPlanNode {
Expr predicates = 1;
bool is_count = 2;
int64 limit = 3;
};
message PlanNode {
oneof node {
VectorANNS vector_anns = 1;
Expr predicates = 2; // deprecated, use query instead.
QueryPlanNode query = 4;
}
repeated int64 output_field_ids = 3;
}
libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/rg.proto 0000664 0001750 0001750 00000002404 15160465216 032453 0 ustar feng feng syntax = "proto3";
package milvus.proto.rg;
option go_package = "github.com/milvus-io/milvus-proto/go-api/v2/rgpb";
option java_multiple_files = true;
option java_package = "io.milvus.grpc";
option java_outer_classname = "ResourceGroupProto";
option java_generate_equals_and_hash = true;
option csharp_namespace = "Milvus.Client.Grpc";
message ResourceGroupLimit {
int32 node_num = 1;
// preserve for other limit.
}
message ResourceGroupTransfer {
string resource_group = 1; // resource groups can be transfered with current resource group.
// preserve for other option, such as weight, priority or affinity setup.
}
message ResourceGroupConfig {
ResourceGroupLimit requests = 1; // requests node num in resource group, if node num is less than requests.nodeNum, it will be transfer from other resource group.
ResourceGroupLimit limits = 2; // limited node num in resource group, if node num is more than limits.nodeNum, it will be transfer to other resource group.
repeated ResourceGroupTransfer transfer_from = 3; // missing node should be transfer from given resource group at high priority in repeated list.
repeated ResourceGroupTransfer transfer_to = 4; // redundant node should be transfer to given resource group at high priority in repeated list.
}
././@LongLink 0000644 0000000 0000000 00000000146 00000000000 011604 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/feder.proto libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/feder.prot0000664 0001750 0001750 00000001640 15160465216 032752 0 ustar feng feng syntax = "proto3";
package milvus.proto.feder;
option go_package = "github.com/milvus-io/milvus-proto/go-api/v2/federpb";
import "common.proto";
message SegmentIndexData {
int64 segmentID = 1;
string index_data = 2; // data from knownwhere
}
message FederSegmentSearchResult {
int64 segmentID = 1;
string visit_info = 2;
}
message ListIndexedSegmentRequest {
common.MsgBase base = 1;
string collection_name = 2;
string index_name = 3;
}
message ListIndexedSegmentResponse {
common.Status status = 1;
repeated int64 segmentIDs = 2;
}
message DescribeSegmentIndexDataRequest {
common.MsgBase base = 1;
string collection_name = 2;
string index_name = 3;
repeated int64 segmentsIDs = 4;
}
message DescribeSegmentIndexDataResponse {
common.Status status = 1;
// segmentID => segmentIndexData
map index_data = 2;
repeated common.KeyValuePair index_params = 3;
}
././@LongLink 0000644 0000000 0000000 00000000147 00000000000 011605 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/common.proto libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/vector-db/common.pro0000664 0001750 0001750 00000026122 15160465216 032773 0 ustar feng feng syntax = "proto3";
package milvus.proto.common;
option go_package="github.com/milvus-io/milvus-proto/go-api/v2/commonpb";
option java_multiple_files = true;
option java_package = "io.milvus.grpc";
option java_outer_classname = "CommonProto";
option java_generate_equals_and_hash = true;
option csharp_namespace = "Milvus.Client.Grpc";
import "google/protobuf/descriptor.proto";
// Deprecated
enum ErrorCode {
option deprecated = true;
Success = 0;
UnexpectedError = 1;
ConnectFailed = 2;
PermissionDenied = 3;
CollectionNotExists = 4;
IllegalArgument = 5;
IllegalDimension = 7;
IllegalIndexType = 8;
IllegalCollectionName = 9;
IllegalTOPK = 10;
IllegalRowRecord = 11;
IllegalVectorID = 12;
IllegalSearchResult = 13;
FileNotFound = 14;
MetaFailed = 15;
CacheFailed = 16;
CannotCreateFolder = 17;
CannotCreateFile = 18;
CannotDeleteFolder = 19;
CannotDeleteFile = 20;
BuildIndexError = 21;
IllegalNLIST = 22;
IllegalMetricType = 23;
OutOfMemory = 24;
IndexNotExist = 25;
EmptyCollection = 26;
UpdateImportTaskFailure = 27;
CollectionNameNotFound = 28;
CreateCredentialFailure = 29;
UpdateCredentialFailure = 30;
DeleteCredentialFailure = 31;
GetCredentialFailure = 32;
ListCredUsersFailure = 33;
GetUserFailure = 34;
CreateRoleFailure = 35;
DropRoleFailure = 36;
OperateUserRoleFailure = 37;
SelectRoleFailure = 38;
SelectUserFailure = 39;
SelectResourceFailure = 40;
OperatePrivilegeFailure = 41;
SelectGrantFailure = 42;
RefreshPolicyInfoCacheFailure = 43;
ListPolicyFailure = 44;
NotShardLeader = 45;
NoReplicaAvailable = 46;
SegmentNotFound = 47;
ForceDeny = 48;
RateLimit = 49;
NodeIDNotMatch = 50;
UpsertAutoIDTrue = 51;
InsufficientMemoryToLoad = 52;
MemoryQuotaExhausted = 53;
DiskQuotaExhausted = 54;
TimeTickLongDelay = 55;
NotReadyServe = 56;
// Coord is switching from standby mode to active mode
NotReadyCoordActivating = 57;
// Service availability.
// NA: Not Available.
DataCoordNA = 100;
// internal error code.
DDRequestRace = 1000;
}
enum IndexState {
IndexStateNone = 0;
Unissued = 1;
InProgress = 2;
Finished = 3;
Failed = 4;
Retry = 5;
}
enum SegmentState {
SegmentStateNone = 0;
NotExist = 1;
Growing = 2;
Sealed = 3;
Flushed = 4;
Flushing = 5;
Dropped = 6;
Importing = 7;
}
message Status {
ErrorCode error_code = 1 [deprecated=true];
string reason = 2;
int32 code = 3;
bool retriable = 4;
string detail = 5;
map extra_info = 6;
}
message KeyValuePair {
string key = 1;
string value = 2;
}
message KeyDataPair {
string key = 1;
bytes data = 2;
}
message Blob {
bytes value = 1;
}
enum PlaceholderType {
None = 0;
BinaryVector = 100;
FloatVector = 101;
Float16Vector = 102;
BFloat16Vector = 103;
SparseFloatVector = 104;
Int64 = 5;
VarChar = 21;
}
message PlaceholderValue {
string tag = 1;
PlaceholderType type = 2;
// values is a 2d-array of nq rows, every row contains a query vector.
// for dense vector, all rows are of the same length; for sparse vector,
// the length of each row may vary depending on their number of non-zeros.
repeated bytes values = 3;
}
message PlaceholderGroup {
repeated PlaceholderValue placeholders = 1;
}
message Address {
string ip = 1;
int64 port = 2;
}
enum MsgType {
Undefined = 0;
/* DEFINITION REQUESTS: COLLECTION */
CreateCollection = 100;
DropCollection = 101;
HasCollection = 102;
DescribeCollection = 103;
ShowCollections = 104;
GetSystemConfigs = 105;
LoadCollection = 106;
ReleaseCollection = 107;
CreateAlias = 108;
DropAlias = 109;
AlterAlias = 110;
AlterCollection = 111;
RenameCollection = 112;
DescribeAlias = 113;
ListAliases = 114;
/* DEFINITION REQUESTS: PARTITION */
CreatePartition = 200;
DropPartition = 201;
HasPartition = 202;
DescribePartition = 203;
ShowPartitions = 204;
LoadPartitions = 205;
ReleasePartitions = 206;
/* DEFINE REQUESTS: SEGMENT */
ShowSegments = 250;
DescribeSegment = 251;
LoadSegments = 252;
ReleaseSegments = 253;
HandoffSegments = 254;
LoadBalanceSegments = 255;
DescribeSegments = 256;
FederListIndexedSegment = 257;
FederDescribeSegmentIndexData = 258;
/* DEFINITION REQUESTS: INDEX */
CreateIndex = 300;
DescribeIndex = 301;
DropIndex = 302;
GetIndexStatistics = 303;
AlterIndex = 304;
/* MANIPULATION REQUESTS */
Insert = 400;
Delete = 401;
Flush = 402;
ResendSegmentStats = 403;
Upsert = 404;
/* QUERY */
Search = 500;
SearchResult = 501;
GetIndexState = 502;
GetIndexBuildProgress = 503;
GetCollectionStatistics = 504;
GetPartitionStatistics = 505;
Retrieve = 506;
RetrieveResult = 507;
WatchDmChannels = 508;
RemoveDmChannels = 509;
WatchQueryChannels = 510;
RemoveQueryChannels = 511;
SealedSegmentsChangeInfo = 512;
WatchDeltaChannels = 513;
GetShardLeaders = 514;
GetReplicas = 515;
UnsubDmChannel = 516;
GetDistribution = 517;
SyncDistribution = 518;
/* DATA SERVICE */
SegmentInfo = 600;
SystemInfo = 601;
GetRecoveryInfo = 602;
GetSegmentState = 603;
/* SYSTEM CONTROL */
TimeTick = 1200;
QueryNodeStats = 1201; // GOOSE TODO: Remove kQueryNodeStats
LoadIndex = 1202;
RequestID = 1203;
RequestTSO = 1204;
AllocateSegment = 1205;
SegmentStatistics = 1206;
SegmentFlushDone = 1207;
DataNodeTt = 1208;
Connect = 1209;
ListClientInfos = 1210;
AllocTimestamp = 1211;
/* Credential */
CreateCredential = 1500;
GetCredential = 1501;
DeleteCredential = 1502;
UpdateCredential = 1503;
ListCredUsernames = 1504;
/* RBAC */
CreateRole = 1600;
DropRole = 1601;
OperateUserRole = 1602;
SelectRole = 1603;
SelectUser = 1604;
SelectResource = 1605;
OperatePrivilege = 1606;
SelectGrant = 1607;
RefreshPolicyInfoCache = 1608;
ListPolicy = 1609;
/* Resource group */
CreateResourceGroup = 1700;
DropResourceGroup = 1701;
ListResourceGroups = 1702;
DescribeResourceGroup = 1703;
TransferNode = 1704;
TransferReplica = 1705;
UpdateResourceGroups = 1706;
/* Database group */
CreateDatabase = 1801;
DropDatabase = 1802;
ListDatabases = 1803;
}
message MsgBase {
MsgType msg_type = 1;
int64 msgID = 2;
uint64 timestamp = 3;
int64 sourceID = 4;
int64 targetID = 5;
map properties = 6;
ReplicateInfo replicateInfo = 7;
}
message ReplicateInfo {
bool isReplicate = 1;
uint64 msgTimestamp = 2;
}
enum DslType {
Dsl = 0;
BoolExprV1 = 1;
}
// Don't Modify This. @czs
message MsgHeader {
common.MsgBase base = 1;
}
// Don't Modify This. @czs
message DMLMsgHeader {
common.MsgBase base = 1;
string shardName = 2;
}
enum CompactionState {
UndefiedState = 0;
Executing = 1;
Completed = 2;
}
enum ConsistencyLevel {
Strong = 0;
Session = 1; // default in PyMilvus
Bounded = 2;
Eventually = 3;
Customized = 4; // Users pass their own `guarantee_timestamp`.
}
enum ImportState {
ImportPending = 0; // the task in in pending list of rootCoord, waiting to be executed
ImportFailed = 1; // the task failed for some reason, get detail reason from GetImportStateResponse.infos
ImportStarted = 2; // the task has been sent to datanode to execute
ImportPersisted = 5; // all data files have been parsed and all meta data already persisted, ready to be flushed.
ImportFlushed = 8; // all segments are successfully flushed.
ImportCompleted = 6; // all indexes are successfully built and segments are able to be compacted as normal.
ImportFailedAndCleaned = 7; // the task failed and all segments it generated are cleaned up.
}
enum ObjectType {
Collection = 0;
Global = 1;
User = 2;
}
enum ObjectPrivilege {
PrivilegeAll = 0;
PrivilegeCreateCollection = 1;
PrivilegeDropCollection = 2;
PrivilegeDescribeCollection = 3;
PrivilegeShowCollections = 4;
PrivilegeLoad = 5;
PrivilegeRelease = 6;
PrivilegeCompaction = 7;
PrivilegeInsert = 8;
PrivilegeDelete = 9;
PrivilegeGetStatistics = 10;
PrivilegeCreateIndex = 11;
PrivilegeIndexDetail = 12;
PrivilegeDropIndex = 13;
PrivilegeSearch = 14;
PrivilegeFlush = 15;
PrivilegeQuery = 16;
PrivilegeLoadBalance = 17;
PrivilegeImport = 18;
PrivilegeCreateOwnership = 19;
PrivilegeUpdateUser = 20;
PrivilegeDropOwnership = 21;
PrivilegeSelectOwnership = 22;
PrivilegeManageOwnership = 23;
PrivilegeSelectUser = 24;
PrivilegeUpsert = 25;
PrivilegeCreateResourceGroup = 26;
PrivilegeDropResourceGroup = 27;
PrivilegeDescribeResourceGroup = 28;
PrivilegeListResourceGroups = 29;
PrivilegeTransferNode = 30;
PrivilegeTransferReplica = 31;
PrivilegeGetLoadingProgress = 32;
PrivilegeGetLoadState = 33;
PrivilegeRenameCollection = 34;
PrivilegeCreateDatabase = 35;
PrivilegeDropDatabase = 36;
PrivilegeListDatabases = 37;
PrivilegeFlushAll = 38;
PrivilegeCreatePartition = 39;
PrivilegeDropPartition = 40;
PrivilegeShowPartitions = 41;
PrivilegeHasPartition = 42;
PrivilegeGetFlushState = 43;
PrivilegeCreateAlias = 44;
PrivilegeDropAlias = 45;
PrivilegeDescribeAlias = 46;
PrivilegeListAliases = 47;
PrivilegeUpdateResourceGroups = 48;
PrivilegeAlterDatabase = 49;
}
message PrivilegeExt {
ObjectType object_type = 1;
ObjectPrivilege object_privilege = 2;
int32 object_name_index = 3;
int32 object_name_indexs = 4;
}
extend google.protobuf.MessageOptions {
PrivilegeExt privilege_ext_obj = 1001;
}
enum StateCode {
Initializing = 0;
Healthy = 1;
Abnormal = 2;
StandBy = 3;
Stopping = 4;
}
enum LoadState {
LoadStateNotExist = 0;
LoadStateNotLoad = 1;
LoadStateLoading = 2;
LoadStateLoaded = 3;
}
message SegmentStats {
int64 SegmentID = 1;
int64 NumRows = 2;
}
message ClientInfo {
// sdk_type can be `python`, `golang`, `nodejs` and etc. It's not proper to make `sdk_type` an
// enumerate type, since we cannot always update the enum value everytime when newly sdk is supported.
string sdk_type = 1;
string sdk_version = 2;
string local_time = 3;
string user = 4;
string host = 5;
// reserved for newly-added feature if necessary.
map reserved = 6;
}
message ServerInfo {
string build_tags = 1;
string build_time = 2;
string git_commit = 3;
string go_version = 4;
string deploy_mode = 5;
// reserved for newly-added feature if necessary.
map reserved = 6;
}
// NodeInfo is used to describe the node information.
message NodeInfo {
int64 node_id = 1;
string address = 2;
string hostname = 3;
}
libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/document-service/ 0000775 0001750 0001750 00000000000 15160465216 032345 5 ustar feng feng ././@LongLink 0000644 0000000 0000000 00000000160 00000000000 011600 L ustar root root libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/document-service/document.proto libkylin-ai-system-private/ai-proto/protocols/usr/share/kylin-ai/protocols/grpc/document-service/doc0000664 0001750 0001750 00000001311 15160465216 033031 0 ustar feng feng syntax = "proto3";
package document;
// 定义服务
service DocumentService {
rpc SyncChunkDocument(ChunkRequest) returns (ChunkedResponse);
rpc AsyncChunkDocument(ChunkRequest) returns (ChunkedResponse);
rpc StreamChunkDocument(ChunkRequest) returns (stream ChunkedResponse);
}
// 定义请求消息
message ChunkRequest {
repeated string file_paths = 1;
repeated string chunker_types = 2;
repeated int32 chunk_sizes = 3;
repeated string callback_data = 4;
}
message ResponseData {
string file_path = 1;
repeated string chunks = 2;
}
// 定义同步分块响应消息
message ChunkedResponse {
repeated ResponseData response_data = 1;
}
libkylin-ai-system-private/ai-proto/CMakeLists.txt 0000664 0001750 0001750 00000003146 15160465216 021202 0 ustar feng feng cmake_minimum_required(VERSION 3.12)
project(KylinAiProto VERSION 1.0.0)
include(GNUInstallDirs)
add_library(KylinAiProto INTERFACE)
install(TARGETS KylinAiProto
EXPORT KylinAiProto_Targets
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
include(CMakePackageConfigHelpers)
write_basic_package_version_file("KylinAiProtoConfigVersion.cmake"
VERSION ${PROJECT_VERSION}
COMPATIBILITY SameMajorVersion)
configure_package_config_file(
"${PROJECT_SOURCE_DIR}/cmake/KylinAiProtoConfig.cmake.in"
"${PROJECT_BINARY_DIR}/KylinAiProtoConfig.cmake"
INSTALL_DESTINATION
${CMAKE_INSTALL_DATAROOTDIR}/${PROJECT_NAME}/cmake)
configure_file("${PROJECT_SOURCE_DIR}/cmake/KylinAiProtoFunctions.cmake"
${CMAKE_CURRENT_BINARY_DIR}/KylinAiProtoFunctions.cmake
COPYONLY)
configure_file("${PROJECT_SOURCE_DIR}/cmake/KylinAiProtoGenerate.cmake"
${CMAKE_CURRENT_BINARY_DIR}/KylinAiProtoGenerate.cmake
COPYONLY)
install(EXPORT KylinAiProto_Targets
FILE KylinAiProtoTargets.cmake
NAMESPACE KylinAiProto::
DESTINATION ${CMAKE_INSTALL_DATAROOTDIR}/${PROJECT_NAME}/cmake)
install(FILES "${PROJECT_BINARY_DIR}/KylinAiProtoConfig.cmake"
"${PROJECT_BINARY_DIR}/KylinAiProtoConfigVersion.cmake"
"${CMAKE_CURRENT_BINARY_DIR}/KylinAiProtoFunctions.cmake"
"${CMAKE_CURRENT_BINARY_DIR}/KylinAiProtoGenerate.cmake"
DESTINATION ${CMAKE_INSTALL_DATAROOTDIR}/${PROJECT_NAME}/cmake)
install(DIRECTORY protocols/usr/ DESTINATION /usr)
libkylin-ai-system-private/ai-proto/cmake/ 0000775 0001750 0001750 00000000000 15160465216 017516 5 ustar feng feng libkylin-ai-system-private/ai-proto/cmake/KylinAiProtoFunctions.cmake 0000664 0001750 0001750 00000003545 15160465216 025004 0 ustar feng feng function(kylin_ai_generate_gdbus_proto_code RETURNED_GDBUS_GLUE_FILES) #REQUIRED_GLIB_MIN_VERSION #XMLFILENAMES)
set(REQUIRED_GLIB_MIN_VERSION)
set(XML_FILE_NAMES)
set(XML_FILES)
set(GLUE_FILES)
foreach(ARG ${ARGN})
if(ARG MATCHES "^--glib-min-required=([0-9]+)\\.([0-9]+)$")
set(REQUIRED_GLIB_MIN_VERSION ${ARG})
message(STATUS "gdbus-codegen: --glib-min-required=${ARG}")
else()
list(APPEND XML_FILE_NAMES ${ARG})
endif()
endforeach()
foreach(XML_FILE_NAME ${XML_FILE_NAMES})
set(XML_FILE /usr/share/kylin-ai/protocols/gdbus/${XML_FILE_NAME}.xml)
if(NOT EXISTS ${XML_FILE})
message(FATAL_ERROR "gdbus-codegen: Unable to find XML file: ${XML_FILE_NAME}")
else()
list(APPEND XML_FILES ${XML_FILE})
endif()
endforeach()
foreach(PROTO_FILE ${XML_FILES})
get_filename_component(PROTO_FILE_NAME ${PROTO_FILE} NAME)
string(REPLACE ".xml" "" PROTO_FILE_NAME ${PROTO_FILE_NAME})
execute_process(
COMMAND /usr/bin/gdbus-codegen ${REQUIRED_GLIB_MIN_VERSION} --output-directory=${CMAKE_CURRENT_BINARY_DIR}
--generate-c-code=${PROTO_FILE_NAME}glue --interface-prefix=com.kylin ${PROTO_FILE}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
RESULT_VARIABLE GDBUS_RESULT
)
if(NOT GDBUS_RESULT EQUAL 0)
message(FATAL_ERROR "gdbus-codegen command failed with error code: ${GDBUS_RESULT}")
else()
list(APPEND GLUE_FILES "${CMAKE_CURRENT_BINARY_DIR}/${PROTO_FILE_NAME}glue.c")
list(APPEND GLUE_FILES "${CMAKE_CURRENT_BINARY_DIR}/${PROTO_FILE_NAME}glue.h")
endif()
endforeach()
include_directories(${CMAKE_CURRENT_BINARY_DIR})
set(${RETURNED_GDBUS_GLUE_FILES} ${GLUE_FILES} PARENT_SCOPE)
endfunction() libkylin-ai-system-private/ai-proto/cmake/KylinAiProtoConfig.cmake.in 0000664 0001750 0001750 00000000402 15160465216 024633 0 ustar feng feng @PACKAGE_INIT@
include("${CMAKE_CURRENT_LIST_DIR}/@PROJECT_NAME@Targets.cmake")
include("${CMAKE_CURRENT_LIST_DIR}/@PROJECT_NAME@Functions.cmake")
include("${CMAKE_CURRENT_LIST_DIR}/@PROJECT_NAME@Generate.cmake")
check_required_components("@PROJECT_NAME@")
libkylin-ai-system-private/ai-proto/cmake/KylinAiProtoGenerate.cmake 0000664 0001750 0001750 00000010721 15160465216 024560 0 ustar feng feng # protoc -I=$SRC_DIR --grpc_out=$DST_DIR --plugin=protoc-gen-grpc=`which grpc_cpp_plugin` $SRC_DIR/xxx.proto 生成服务端代码
# protoc -I=$SRC_DIR --cpp_out=$DST_DIR $SRC_DIR/xxx.proto
# 调用解析文件函数前需要引用protobuf，find_package(Protobuf REQUIRED)
message(STATUS "using protoc: ${PROTOBUF_PROTOC_EXECUTABLE}")
# 调用解析文件函数前需要设置存放生成文件的路径，未设置使用默认值 CMAKE_CURRENT_BINARY_DIR
if(DEFINED GRPC_GENERATE_FILES_DIR)
message("grpc output dir: ${GRPC_GENERATE_FILES_DIR}")
else()
set(GRPC_GENERATE_FILES_DIR ${CMAKE_CURRENT_BINARY_DIR})
message("grpc output dir not set, use default dir: ${GRPC_GENERATE_FILES_DIR}")
endif()
# 示例：kylin_ai_generate_grpc_proto_source(RETURNED_GRPC_SOURCE_FILES vector-db common feder msg rg schema milvus)
function(kylin_ai_generate_grpc_proto_source RETURNED_GRPC_SOURCE_FILES PATH)
set(PROTO_FILE_NAMES)
set(PROTO_FILES)
set(GENERATE_PROTO_FILES)
set(SRC_DIR /usr/share/kylin-ai/protocols/grpc/${PATH})
# ARGN 是函数传入以外的参数
foreach(ARG ${ARGN})
list(APPEND PROTO_FILE_NAMES ${ARG})
endforeach()
foreach(PROTO_FILE_NAME ${PROTO_FILE_NAMES})
set(PROTO_FILE ${SRC_DIR}/${PROTO_FILE_NAME}.proto)
if(NOT EXISTS ${PROTO_FILE})
message(FATAL_ERROR "grpc-proto-gen: Unable to find proto file: ${PROTO_FILE}")
else()
list(APPEND PROTO_FILES ${PROTO_FILE})
endif()
endforeach()
foreach(PROTO_FILE ${PROTO_FILES})
get_filename_component(PROTO_FILE_NAME ${PROTO_FILE} NAME)
string(REPLACE ".proto" "" PROTO_FILE_NAME ${PROTO_FILE_NAME})
add_custom_command(
OUTPUT ${GRPC_GENERATE_FILES_DIR}/${PROTO_FILE_NAME}.pb.cc
${GRPC_GENERATE_FILES_DIR}/${PROTO_FILE_NAME}.pb.h
DEPENDS ${PROTO_FILE}
${PROTOBUF_PROTOC_EXECUTABLE}
COMMAND ${PROTOBUF_PROTOC_EXECUTABLE}
ARGS --cpp_out ${GRPC_GENERATE_FILES_DIR}
-I ${SRC_DIR}
${PROTO_FILE}
)
list(APPEND GENERATE_PROTO_FILES "${GRPC_GENERATE_FILES_DIR}/${PROTO_FILE_NAME}.pb.cc")
list(APPEND GENERATE_PROTO_FILES "${GRPC_GENERATE_FILES_DIR}/${PROTO_FILE_NAME}.pb.h")
endforeach()
include_directories(${GRPC_GENERATE_FILES_DIR})
set(${RETURNED_GRPC_SOURCE_FILES} ${GENERATE_PROTO_FILES} PARENT_SCOPE)
endfunction(kylin_ai_generate_grpc_proto_source RETURNED_GRPC_SOURCE_FILES)
function(kylin_ai_generate_grpc_proto_service RETURNED_GRPC_SERVICE_FILES PATH)
find_program(GRPC_CPP_PLUGIN grpc_cpp_plugin)
message(STATUS "using protoc: ${GRPC_CPP_PLUGIN}")
set(PROTO_FILE_NAMES)
set(PROTO_FILES)
set(GENERATE_PROTO_FILES)
set(SRC_DIR /usr/share/kylin-ai/protocols/grpc/${PATH})
# ARGN 是函数传入以外的参数
foreach(ARG ${ARGN})
list(APPEND PROTO_FILE_NAMES ${ARG})
endforeach()
foreach(PROTO_FILE_NAME ${PROTO_FILE_NAMES})
set(PROTO_FILE ${SRC_DIR}/${PROTO_FILE_NAME}.proto)
if(NOT EXISTS ${PROTO_FILE})
message(FATAL_ERROR "grpc-proto-gen: Unable to find proto file: ${PROTO_FILE}")
else()
list(APPEND PROTO_FILES ${PROTO_FILE})
endif()
endforeach()
foreach(PROTO_FILE ${PROTO_FILES})
get_filename_component(PROTO_FILE_NAME ${PROTO_FILE} NAME)
string(REPLACE ".proto" "" PROTO_FILE_NAME ${PROTO_FILE_NAME})
add_custom_command(
OUTPUT ${GRPC_GENERATE_FILES_DIR}/${PROTO_FILE_NAME}.grpc.pb.cc
${GRPC_GENERATE_FILES_DIR}/${PROTO_FILE_NAME}.grpc.pb.h
DEPENDS ${PROTO_FILE}
${PROTOBUF_PROTOC_EXECUTABLE}
${GRPC_CPP_PLUGIN}
COMMAND ${PROTOBUF_PROTOC_EXECUTABLE}
ARGS --grpc_out ${GRPC_GENERATE_FILES_DIR}
-I ${SRC_DIR}
--plugin=protoc-gen-grpc=${GRPC_CPP_PLUGIN}
${PROTO_FILE}
)
list(APPEND GENERATE_PROTO_FILES "${GRPC_GENERATE_FILES_DIR}/${PROTO_FILE_NAME}.grpc.pb.cc")
list(APPEND GENERATE_PROTO_FILES "${GRPC_GENERATE_FILES_DIR}/${PROTO_FILE_NAME}.grpc.pb.h")
endforeach()
include_directories(${GRPC_GENERATE_FILES_DIR})
set(${RETURNED_GRPC_SERVICE_FILES} ${GENERATE_PROTO_FILES} PARENT_SCOPE)
endfunction(kylin_ai_generate_grpc_proto_service RETURNED_GRPC_SERVICE_FILES)
libkylin-ai-system-private/.gitignore 0000775 0001750 0001750 00000001517 15160465216 016703 0 ustar feng feng # This file is used to ignore files which are generated
# ----------------------------------------------------------------------------
*~
*.autosave
*.a
*.core
*.moc
*.o
*.obj
*.orig
*.rej
*.so
*.so.*
*_pch.h.cpp
*_resource.rc
*.qm
.#*
*.*#
core
!core/
tags
.DS_Store
.directory
*.debug
Makefile*
*.prl
*.app
moc_*.cpp
ui_*.h
qrc_*.cpp
Thumbs.db
*.res
*.rc
/.qmake.cache
/.qmake.stash
# qtcreator generated files
*.pro.user*
CMakeLists.txt.user*
# xemacs temporary files
*.flc
# Vim temporary files
.*.swp
# Visual Studio generated files
*.ib_pdb_index
*.idb
*.ilk
*.pdb
*.sln
*.suo
*.vcproj
*vcproj.*.*.user
*.ncb
*.sdf
*.opensdf
*.vcxproj
*vcxproj.*
# MinGW generated files
*.Debug
*.Release
# Python byte code
*.pyc
# Binaries
# --------
*.dll
*.exe
build
.vscode
.cache
.reuse
# third-party
third-party/llama.cpp/common/build-info.cpp